2024-04-22T05:37:48.869 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-22T05:37:48.869 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776 --name teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi --verbose -- /home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776/orig.config.yaml 2024-04-22T05:37:48.906 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T05:37:48.990 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776 branch: reef description: rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} email: ceph-qa@ceph.com first_in_suite: false job_id: '7667776' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: reef ceph: conf: client: debug cephsqlite: 20 debug client: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 3 ms inject socket failures: 1500 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - POOL_APP_NOT_ENABLED - do not have an application enabled sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: c3a345d31ea77c01771428ab80e1cf32c9b05875 owner: scheduled_teuthology@teuthology priority: 931 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 7353 sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d sleep_before_teardown: 0 subset: 3333/100000 suite: rados suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_c3a345d31ea77c01771428ab80e1cf32c9b05875/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: c3a345d31ea77c01771428ab80e1cf32c9b05875 targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBDc0EB6DtneaHuiYu9Y1eM3NOer0L1aCLtyVQGlyCL8v3dFJlp8ejgZD8QcxGvUFycLqv3OBt4amm0SQ7XLJPM= smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI82nsOcFRNQibKSdwEJYJV0XxWtfkzblKfQ8RBRWWT1NjFKhQ5xDBnL8q9kSPxxvQ+ptRhLtBiCYCFDu+W7GMI= tasks: - install: extra_system_packages: deb: - sqlite3 rpm: - sqlite-devel - ceph: null - exec: client.0: - ceph osd pool create cephsqlite - ceph auth get-or-create client.libcephsqlite mon 'profile simple-rados-client-with-blocklist' osd 'allow rwx pool=cephsqlite' >> /etc/ceph/ceph.keyring - exec: client.0: - ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr - workunit: clients: client.0: - rados/test_libcephsqlite.sh cephsqlite env: CEPH_ARGS: --id libcephsqlite --no-log-to-stderr teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-21_22:00:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-22T05:37:48.990 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_c3a345d31ea77c01771428ab80e1cf32c9b05875/qa; will attempt to use it 2024-04-22T05:37:48.991 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_c3a345d31ea77c01771428ab80e1cf32c9b05875/qa/tasks 2024-04-22T05:37:48.991 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-22T05:37:48.993 INFO:teuthology.task.internal:Checking packages... 2024-04-22T05:37:49.018 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '646b8c7d293d8ec31a40c69f7b3237b19c1be24d' 2024-04-22T05:37:49.018 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T05:37:49.018 INFO:teuthology.packaging:ref: None 2024-04-22T05:37:49.018 INFO:teuthology.packaging:tag: None 2024-04-22T05:37:49.018 INFO:teuthology.packaging:branch: reef 2024-04-22T05:37:49.018 INFO:teuthology.packaging:sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-22T05:37:49.018 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef 2024-04-22T05:37:49.258 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1237-g646b8c7d-1jammy 2024-04-22T05:37:49.259 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-22T05:37:49.266 INFO:teuthology.task.internal:no buildpackages task found 2024-04-22T05:37:49.266 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-22T05:37:49.286 INFO:teuthology.task.internal:Saving configuration 2024-04-22T05:37:49.300 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-22T05:37:49.306 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T05:37:49.332 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 05:32:09.627231', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDb/nl6fDdJLIXByGHa3VaVEryueaAz6Cu40GT9WQDwK'} 2024-04-22T05:37:49.355 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 05:32:09.629064', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGkIUgRDvcefjG4SOBAlwanNuhAmBnT8yGpXBKshas0l'} 2024-04-22T05:37:49.355 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-22T05:37:49.361 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-04-22T05:37:49.361 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-04-22T05:37:49.361 INFO:teuthology.run_tasks:Running task console_log... 2024-04-22T05:37:49.436 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6007329280>, signals=[15]) 2024-04-22T05:37:49.436 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-22T05:37:49.443 INFO:teuthology.task.internal:Opening connections... 2024-04-22T05:37:49.443 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-04-22T05:37:49.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T05:37:49.527 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-04-22T05:37:49.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T05:37:49.605 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-22T05:37:49.611 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-04-22T05:37:49.616 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-04-22T05:37:49.617 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-04-22T05:37:49.663 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-22T05:37:49.663 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="Ubuntu" 2024-04-22T05:37:49.663 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="22.04" 2024-04-22T05:37:49.663 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-22T05:37:49.663 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_CODENAME=jammy 2024-04-22T05:37:49.663 INFO:teuthology.orchestra.run.smithi040.stdout:ID=ubuntu 2024-04-22T05:37:49.664 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE=debian 2024-04-22T05:37:49.664 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-22T05:37:49.664 INFO:teuthology.orchestra.run.smithi040.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-22T05:37:49.664 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-22T05:37:49.664 INFO:teuthology.orchestra.run.smithi040.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-22T05:37:49.664 INFO:teuthology.orchestra.run.smithi040.stdout:UBUNTU_CODENAME=jammy 2024-04-22T05:37:49.664 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-04-22T05:37:49.683 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-04-22T05:37:49.688 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-04-22T05:37:49.688 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-04-22T05:37:49.734 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-22T05:37:49.734 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Ubuntu" 2024-04-22T05:37:49.734 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="22.04" 2024-04-22T05:37:49.735 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-22T05:37:49.735 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_CODENAME=jammy 2024-04-22T05:37:49.735 INFO:teuthology.orchestra.run.smithi062.stdout:ID=ubuntu 2024-04-22T05:37:49.735 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE=debian 2024-04-22T05:37:49.735 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-22T05:37:49.735 INFO:teuthology.orchestra.run.smithi062.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-22T05:37:49.735 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-22T05:37:49.735 INFO:teuthology.orchestra.run.smithi062.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-22T05:37:49.735 INFO:teuthology.orchestra.run.smithi062.stdout:UBUNTU_CODENAME=jammy 2024-04-22T05:37:49.735 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-04-22T05:37:49.758 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-22T05:37:49.769 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-22T05:37:49.774 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-22T05:37:49.775 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-04-22T05:37:49.777 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-04-22T05:37:49.782 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-22T05:37:49.787 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-22T05:37:49.788 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-04-22T05:37:49.825 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-04-22T05:37:49.842 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-22T05:37:49.895 INFO:teuthology.run_tasks:Running task kernel... 2024-04-22T05:37:49.908 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-22T05:37:49.908 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-22T05:37:49.909 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T05:37:49.909 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T05:37:49.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T05:37:49.914 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-04-22T05:37:49.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T05:37:49.915 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-04-22T05:37:49.960 INFO:teuthology.orchestra.run.smithi040.stdout:5.15.0-56-generic 2024-04-22T05:37:49.960 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.15.0-56-generic 2024-04-22T05:37:49.960 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get clean 2024-04-22T05:37:49.963 INFO:teuthology.orchestra.run.smithi062.stdout:5.15.0-56-generic 2024-04-22T05:37:49.963 INFO:teuthology.task.kernel:Running kernel on smithi062: 5.15.0-56-generic 2024-04-22T05:37:49.963 DEBUG:teuthology.orchestra.run.smithi062:> sudo apt-get clean 2024-04-22T05:37:50.095 DEBUG:teuthology.orchestra.run.smithi062:> sudo apt-get update 2024-04-22T05:37:50.112 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get update 2024-04-22T05:37:50.259 INFO:teuthology.orchestra.run.smithi062.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-22T05:37:50.285 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-22T05:37:50.296 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-22T05:37:50.350 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-22T05:37:50.370 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-22T05:37:50.389 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-22T05:37:50.431 INFO:teuthology.orchestra.run.smithi062.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-22T05:37:50.470 INFO:teuthology.orchestra.run.smithi062.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [450 kB] 2024-04-22T05:37:50.545 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,597 kB] 2024-04-22T05:37:50.554 INFO:teuthology.orchestra.run.smithi062.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,381 kB] 2024-04-22T05:37:50.598 INFO:teuthology.orchestra.run.smithi062.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [242 kB] 2024-04-22T05:37:50.602 INFO:teuthology.orchestra.run.smithi062.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-22T05:37:50.603 INFO:teuthology.orchestra.run.smithi062.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-22T05:37:50.603 INFO:teuthology.orchestra.run.smithi062.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,744 kB] 2024-04-22T05:37:50.630 INFO:teuthology.orchestra.run.smithi062.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [294 kB] 2024-04-22T05:37:50.635 INFO:teuthology.orchestra.run.smithi062.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-22T05:37:50.635 INFO:teuthology.orchestra.run.smithi062.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-22T05:37:50.647 INFO:teuthology.orchestra.run.smithi062.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-22T05:37:50.654 INFO:teuthology.orchestra.run.smithi062.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-04-22T05:37:50.656 INFO:teuthology.orchestra.run.smithi062.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-22T05:37:50.656 INFO:teuthology.orchestra.run.smithi062.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-22T05:37:50.656 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [616 kB] 2024-04-22T05:37:50.657 INFO:teuthology.orchestra.run.smithi062.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-22T05:37:50.664 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [302 kB] 2024-04-22T05:37:50.667 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-22T05:37:50.668 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,806 kB] 2024-04-22T05:37:50.670 INFO:teuthology.orchestra.run.smithi062.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-22T05:37:50.674 INFO:teuthology.orchestra.run.smithi062.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-22T05:37:50.688 INFO:teuthology.orchestra.run.smithi040.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-22T05:37:50.688 INFO:teuthology.orchestra.run.smithi040.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [305 kB] 2024-04-22T05:37:50.691 INFO:teuthology.orchestra.run.smithi040.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-22T05:37:50.691 INFO:teuthology.orchestra.run.smithi040.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-22T05:37:50.697 INFO:teuthology.orchestra.run.smithi040.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-22T05:37:50.706 INFO:teuthology.orchestra.run.smithi040.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-22T05:37:50.708 INFO:teuthology.orchestra.run.smithi040.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-22T05:37:50.708 INFO:teuthology.orchestra.run.smithi040.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-22T05:37:50.744 INFO:teuthology.orchestra.run.smithi040.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-22T05:37:50.745 INFO:teuthology.orchestra.run.smithi040.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-22T05:37:50.745 INFO:teuthology.orchestra.run.smithi040.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-22T05:37:50.745 INFO:teuthology.orchestra.run.smithi040.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-22T05:37:50.746 INFO:teuthology.orchestra.run.smithi040.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-22T05:37:50.746 INFO:teuthology.orchestra.run.smithi040.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-22T05:37:50.746 INFO:teuthology.orchestra.run.smithi040.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-22T05:37:50.748 INFO:teuthology.orchestra.run.smithi040.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-22T05:37:50.767 INFO:teuthology.orchestra.run.smithi040.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-22T05:37:50.768 INFO:teuthology.orchestra.run.smithi040.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-22T05:37:50.768 INFO:teuthology.orchestra.run.smithi040.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-22T05:37:50.794 INFO:teuthology.orchestra.run.smithi062.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-22T05:37:50.856 INFO:teuthology.orchestra.run.smithi040.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [450 kB] 2024-04-22T05:37:50.931 INFO:teuthology.orchestra.run.smithi062.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [616 kB] 2024-04-22T05:37:51.128 INFO:teuthology.orchestra.run.smithi062.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,597 kB] 2024-04-22T05:37:51.139 INFO:teuthology.orchestra.run.smithi040.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,381 kB] 2024-04-22T05:37:51.278 INFO:teuthology.orchestra.run.smithi062.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [302 kB] 2024-04-22T05:37:51.292 INFO:teuthology.orchestra.run.smithi062.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-22T05:37:51.293 INFO:teuthology.orchestra.run.smithi062.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,806 kB] 2024-04-22T05:37:51.306 INFO:teuthology.orchestra.run.smithi040.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [242 kB] 2024-04-22T05:37:51.319 INFO:teuthology.orchestra.run.smithi040.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-22T05:37:51.319 INFO:teuthology.orchestra.run.smithi040.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,744 kB] 2024-04-22T05:37:51.354 INFO:teuthology.orchestra.run.smithi062.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-22T05:37:51.356 INFO:teuthology.orchestra.run.smithi062.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [305 kB] 2024-04-22T05:37:51.363 INFO:teuthology.orchestra.run.smithi062.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-22T05:37:51.401 INFO:teuthology.orchestra.run.smithi040.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-22T05:37:51.402 INFO:teuthology.orchestra.run.smithi040.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [294 kB] 2024-04-22T05:37:51.410 INFO:teuthology.orchestra.run.smithi040.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-22T05:37:51.410 INFO:teuthology.orchestra.run.smithi040.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-22T05:37:51.426 INFO:teuthology.orchestra.run.smithi040.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-22T05:37:51.527 INFO:teuthology.orchestra.run.smithi062.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-22T05:37:51.608 INFO:teuthology.orchestra.run.smithi040.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-04-22T05:37:52.015 INFO:teuthology.orchestra.run.smithi040.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-22T05:37:52.021 INFO:teuthology.orchestra.run.smithi040.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-22T05:37:52.035 INFO:teuthology.orchestra.run.smithi040.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-22T05:37:52.035 INFO:teuthology.orchestra.run.smithi040.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-22T05:37:52.038 INFO:teuthology.orchestra.run.smithi040.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-22T05:37:52.111 INFO:teuthology.orchestra.run.smithi062.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-22T05:37:52.223 INFO:teuthology.orchestra.run.smithi062.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-22T05:37:52.243 INFO:teuthology.orchestra.run.smithi062.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-22T05:37:52.244 INFO:teuthology.orchestra.run.smithi062.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-22T05:37:52.247 INFO:teuthology.orchestra.run.smithi062.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-22T05:37:52.248 INFO:teuthology.orchestra.run.smithi062.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-22T05:37:52.248 INFO:teuthology.orchestra.run.smithi062.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-22T05:37:52.249 INFO:teuthology.orchestra.run.smithi062.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-22T05:37:52.253 INFO:teuthology.orchestra.run.smithi062.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-22T05:37:52.257 INFO:teuthology.orchestra.run.smithi062.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-22T05:37:52.422 INFO:teuthology.orchestra.run.smithi062.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-22T05:37:52.504 INFO:teuthology.orchestra.run.smithi062.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-22T05:37:52.615 INFO:teuthology.orchestra.run.smithi062.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-22T05:37:52.670 INFO:teuthology.orchestra.run.smithi062.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-22T05:37:52.702 INFO:teuthology.orchestra.run.smithi062.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-22T05:37:56.737 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 13.1 MB in 2s (6,799 kB/s) 2024-04-22T05:37:57.260 INFO:teuthology.orchestra.run.smithi062.stdout:Fetched 13.1 MB in 3s (5,184 kB/s) 2024-04-22T05:37:58.055 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-22T05:37:58.073 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-22T05:37:58.137 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-22T05:37:58.338 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2024-04-22T05:37:58.339 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2024-04-22T05:37:58.516 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2024-04-22T05:37:58.516 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-22T05:37:58.516 INFO:teuthology.orchestra.run.smithi040.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-22T05:37:58.517 INFO:teuthology.orchestra.run.smithi040.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-22T05:37:58.517 INFO:teuthology.orchestra.run.smithi040.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-22T05:37:58.517 INFO:teuthology.orchestra.run.smithi040.stdout: python2.7-minimal 2024-04-22T05:37:58.517 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-22T05:37:58.518 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2024-04-22T05:37:58.518 INFO:teuthology.orchestra.run.smithi040.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-22T05:37:58.518 INFO:teuthology.orchestra.run.smithi040.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-22T05:37:58.519 INFO:teuthology.orchestra.run.smithi040.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-22T05:37:58.520 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2024-04-22T05:37:58.520 INFO:teuthology.orchestra.run.smithi040.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-22T05:37:58.520 INFO:teuthology.orchestra.run.smithi040.stdout:Recommended packages: 2024-04-22T05:37:58.520 INFO:teuthology.orchestra.run.smithi040.stdout: thermald 2024-04-22T05:37:58.574 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2024-04-22T05:37:58.574 INFO:teuthology.orchestra.run.smithi040.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-22T05:37:58.574 INFO:teuthology.orchestra.run.smithi040.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-22T05:37:58.575 INFO:teuthology.orchestra.run.smithi040.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-22T05:37:58.576 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages will be upgraded: 2024-04-22T05:37:58.577 INFO:teuthology.orchestra.run.smithi040.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-22T05:37:58.636 INFO:teuthology.orchestra.run.smithi040.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-22T05:37:58.636 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 113 MB of archives. 2024-04-22T05:37:58.636 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-22T05:37:58.636 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-22T05:37:58.651 INFO:teuthology.orchestra.run.smithi062.stdout:Reading package lists... 2024-04-22T05:37:58.668 DEBUG:teuthology.orchestra.run.smithi062:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-22T05:37:58.730 INFO:teuthology.orchestra.run.smithi062.stdout:Reading package lists... 2024-04-22T05:37:58.942 INFO:teuthology.orchestra.run.smithi062.stdout:Building dependency tree... 2024-04-22T05:37:58.943 INFO:teuthology.orchestra.run.smithi062.stdout:Reading state information... 2024-04-22T05:37:59.092 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-22T05:37:59.127 INFO:teuthology.orchestra.run.smithi062.stdout:The following packages were automatically installed and are no longer required: 2024-04-22T05:37:59.127 INFO:teuthology.orchestra.run.smithi062.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-22T05:37:59.127 INFO:teuthology.orchestra.run.smithi062.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-22T05:37:59.128 INFO:teuthology.orchestra.run.smithi062.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-22T05:37:59.128 INFO:teuthology.orchestra.run.smithi062.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-22T05:37:59.128 INFO:teuthology.orchestra.run.smithi062.stdout: python2.7-minimal 2024-04-22T05:37:59.128 INFO:teuthology.orchestra.run.smithi062.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-22T05:37:59.129 INFO:teuthology.orchestra.run.smithi062.stdout:The following additional packages will be installed: 2024-04-22T05:37:59.129 INFO:teuthology.orchestra.run.smithi062.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-22T05:37:59.129 INFO:teuthology.orchestra.run.smithi062.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-22T05:37:59.129 INFO:teuthology.orchestra.run.smithi062.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-22T05:37:59.131 INFO:teuthology.orchestra.run.smithi062.stdout:Suggested packages: 2024-04-22T05:37:59.131 INFO:teuthology.orchestra.run.smithi062.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-22T05:37:59.131 INFO:teuthology.orchestra.run.smithi062.stdout:Recommended packages: 2024-04-22T05:37:59.131 INFO:teuthology.orchestra.run.smithi062.stdout: thermald 2024-04-22T05:37:59.181 INFO:teuthology.orchestra.run.smithi062.stdout:The following NEW packages will be installed: 2024-04-22T05:37:59.181 INFO:teuthology.orchestra.run.smithi062.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-22T05:37:59.181 INFO:teuthology.orchestra.run.smithi062.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-22T05:37:59.182 INFO:teuthology.orchestra.run.smithi062.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-22T05:37:59.183 INFO:teuthology.orchestra.run.smithi062.stdout:The following packages will be upgraded: 2024-04-22T05:37:59.184 INFO:teuthology.orchestra.run.smithi062.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-22T05:37:59.197 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-22T05:37:59.365 INFO:teuthology.orchestra.run.smithi062.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-22T05:37:59.365 INFO:teuthology.orchestra.run.smithi062.stdout:Need to get 113 MB of archives. 2024-04-22T05:37:59.365 INFO:teuthology.orchestra.run.smithi062.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-22T05:37:59.365 INFO:teuthology.orchestra.run.smithi062.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-22T05:38:00.096 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-22T05:38:00.096 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-22T05:38:00.096 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-22T05:38:00.398 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-22T05:38:00.441 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-22T05:38:00.794 INFO:teuthology.orchestra.run.smithi062.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-22T05:38:00.845 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 113 MB in 2s (61.0 MB/s) 2024-04-22T05:38:00.998 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-22T05:38:01.152 INFO:teuthology.orchestra.run.smithi062.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-22T05:38:03.027 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-22T05:38:03.032 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-22T05:38:03.170 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:03.341 INFO:teuthology.orchestra.run.smithi062.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-22T05:38:03.342 INFO:teuthology.orchestra.run.smithi062.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-22T05:38:03.415 INFO:teuthology.orchestra.run.smithi062.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-22T05:38:04.724 INFO:teuthology.orchestra.run.smithi062.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-22T05:38:04.861 INFO:teuthology.orchestra.run.smithi062.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-22T05:38:05.277 INFO:teuthology.orchestra.run.smithi062.stdout:Fetched 113 MB in 6s (20.0 MB/s) 2024-04-22T05:38:05.451 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-22T05:38:05.497 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-22T05:38:05.519 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-22T05:38:05.655 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:06.117 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-22T05:38:06.138 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-22T05:38:06.167 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:07.413 INFO:teuthology.orchestra.run.smithi062.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-22T05:38:07.420 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-22T05:38:07.482 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:09.525 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-22T05:38:09.547 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-22T05:38:09.652 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:10.095 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-22T05:38:10.116 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-22T05:38:10.153 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:13.327 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-22T05:38:13.462 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-22T05:38:13.795 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-22T05:38:13.941 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-22T05:38:14.236 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-22T05:38:14.258 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-22T05:38:14.294 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-22T05:38:16.431 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-22T05:38:16.558 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-22T05:38:16.857 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-22T05:38:16.953 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-22T05:38:17.256 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-22T05:38:17.278 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-22T05:38:17.322 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-22T05:38:20.374 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-22T05:38:20.411 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-22T05:38:20.468 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:23.155 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-22T05:38:23.179 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-22T05:38:23.192 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-22T05:38:23.231 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:23.302 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-22T05:38:23.669 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-22T05:38:23.790 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:23.916 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-22T05:38:24.078 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:25.420 INFO:teuthology.orchestra.run.smithi040.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-22T05:38:25.420 INFO:teuthology.orchestra.run.smithi040.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-22T05:38:25.420 INFO:teuthology.orchestra.run.smithi040.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-22T05:38:25.420 INFO:teuthology.orchestra.run.smithi040.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-22T05:38:25.563 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:26.735 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-22T05:38:26.860 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-22T05:38:27.033 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:27.169 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-22T05:38:27.290 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:27.416 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-22T05:38:27.544 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:28.475 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-22T05:38:28.601 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-22T05:38:28.752 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:28.816 INFO:teuthology.orchestra.run.smithi040.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-22T05:38:28.816 INFO:teuthology.orchestra.run.smithi040.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-22T05:38:28.915 INFO:teuthology.orchestra.run.smithi062.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-22T05:38:28.915 INFO:teuthology.orchestra.run.smithi062.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-22T05:38:28.915 INFO:teuthology.orchestra.run.smithi062.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-22T05:38:28.915 INFO:teuthology.orchestra.run.smithi062.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-22T05:38:29.080 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:30.523 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:32.283 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-22T05:38:32.409 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-22T05:38:32.535 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-22T05:38:32.601 INFO:teuthology.orchestra.run.smithi062.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-22T05:38:32.602 INFO:teuthology.orchestra.run.smithi062.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-22T05:38:43.855 INFO:teuthology.orchestra.run.smithi040.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-22T05:38:43.855 INFO:teuthology.orchestra.run.smithi040.stdout:Sourcing file `/etc/default/grub' 2024-04-22T05:38:43.870 INFO:teuthology.orchestra.run.smithi040.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-22T05:38:43.897 INFO:teuthology.orchestra.run.smithi040.stdout:Generating grub configuration file ... 2024-04-22T05:38:44.293 INFO:teuthology.orchestra.run.smithi040.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-22T05:38:44.316 INFO:teuthology.orchestra.run.smithi040.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-22T05:38:44.682 INFO:teuthology.orchestra.run.smithi040.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-22T05:38:44.688 INFO:teuthology.orchestra.run.smithi040.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-22T05:38:44.756 INFO:teuthology.orchestra.run.smithi040.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-22T05:38:44.761 INFO:teuthology.orchestra.run.smithi040.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-22T05:38:44.919 INFO:teuthology.orchestra.run.smithi040.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-22T05:38:44.919 INFO:teuthology.orchestra.run.smithi040.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-22T05:38:44.919 INFO:teuthology.orchestra.run.smithi040.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-22T05:38:44.955 INFO:teuthology.orchestra.run.smithi040.stdout:done 2024-04-22T05:38:45.474 DEBUG:teuthology.orchestra.run.smithi040:> dpkg -s linux-image-generic 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:Package: linux-image-generic 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:Status: install ok installed 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:Priority: optional 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:Section: kernel 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:Installed-Size: 21 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:Maintainer: Ubuntu Kernel Team 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: amd64 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:Source: linux-meta 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:Version: 5.15.0.105.102 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:Recommends: thermald 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:Description: Generic Linux kernel image 2024-04-22T05:38:45.492 INFO:teuthology.orchestra.run.smithi040.stdout: This package will always depend on the latest generic kernel image 2024-04-22T05:38:45.493 INFO:teuthology.orchestra.run.smithi040.stdout: available. 2024-04-22T05:38:45.493 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-22T05:38:45.493 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-22T05:38:45.493 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-22T05:38:45.493 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-22T05:38:45.493 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi040.front.sepia.ceph.com, path=None, version=distro) 2024-04-22T05:38:45.493 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get clean 2024-04-22T05:38:45.667 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get update 2024-04-22T05:38:45.798 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-22T05:38:45.901 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-22T05:38:45.981 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-22T05:38:46.063 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-22T05:38:46.523 INFO:teuthology.orchestra.run.smithi062.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-22T05:38:46.523 INFO:teuthology.orchestra.run.smithi062.stdout:Sourcing file `/etc/default/grub' 2024-04-22T05:38:46.541 INFO:teuthology.orchestra.run.smithi062.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-22T05:38:46.558 INFO:teuthology.orchestra.run.smithi062.stdout:Generating grub configuration file ... 2024-04-22T05:38:46.849 INFO:teuthology.orchestra.run.smithi062.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-22T05:38:46.871 INFO:teuthology.orchestra.run.smithi062.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-22T05:38:47.154 INFO:teuthology.orchestra.run.smithi062.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-22T05:38:47.161 INFO:teuthology.orchestra.run.smithi062.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-22T05:38:47.187 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-22T05:38:47.206 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-22T05:38:47.230 INFO:teuthology.orchestra.run.smithi062.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-22T05:38:47.235 INFO:teuthology.orchestra.run.smithi062.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-22T05:38:47.266 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-22T05:38:47.402 INFO:teuthology.orchestra.run.smithi062.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-22T05:38:47.402 INFO:teuthology.orchestra.run.smithi062.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-22T05:38:47.402 INFO:teuthology.orchestra.run.smithi062.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-22T05:38:47.442 INFO:teuthology.orchestra.run.smithi062.stdout:done 2024-04-22T05:38:47.479 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2024-04-22T05:38:47.480 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2024-04-22T05:38:47.663 INFO:teuthology.orchestra.run.smithi040.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-22T05:38:47.663 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2024-04-22T05:38:47.663 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-22T05:38:47.663 INFO:teuthology.orchestra.run.smithi040.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-22T05:38:47.663 INFO:teuthology.orchestra.run.smithi040.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-22T05:38:47.664 INFO:teuthology.orchestra.run.smithi040.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-22T05:38:47.664 INFO:teuthology.orchestra.run.smithi040.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-22T05:38:47.664 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-22T05:38:47.707 INFO:teuthology.orchestra.run.smithi040.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-22T05:38:47.709 DEBUG:teuthology.orchestra.run.smithi040:> dpkg -s linux-image-generic 2024-04-22T05:38:47.725 INFO:teuthology.orchestra.run.smithi040.stdout:Package: linux-image-generic 2024-04-22T05:38:47.725 INFO:teuthology.orchestra.run.smithi040.stdout:Status: install ok installed 2024-04-22T05:38:47.725 INFO:teuthology.orchestra.run.smithi040.stdout:Priority: optional 2024-04-22T05:38:47.725 INFO:teuthology.orchestra.run.smithi040.stdout:Section: kernel 2024-04-22T05:38:47.726 INFO:teuthology.orchestra.run.smithi040.stdout:Installed-Size: 21 2024-04-22T05:38:47.726 INFO:teuthology.orchestra.run.smithi040.stdout:Maintainer: Ubuntu Kernel Team 2024-04-22T05:38:47.726 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: amd64 2024-04-22T05:38:47.726 INFO:teuthology.orchestra.run.smithi040.stdout:Source: linux-meta 2024-04-22T05:38:47.726 INFO:teuthology.orchestra.run.smithi040.stdout:Version: 5.15.0.105.102 2024-04-22T05:38:47.726 INFO:teuthology.orchestra.run.smithi040.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-22T05:38:47.726 INFO:teuthology.orchestra.run.smithi040.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-22T05:38:47.726 INFO:teuthology.orchestra.run.smithi040.stdout:Recommends: thermald 2024-04-22T05:38:47.726 INFO:teuthology.orchestra.run.smithi040.stdout:Description: Generic Linux kernel image 2024-04-22T05:38:47.726 INFO:teuthology.orchestra.run.smithi040.stdout: This package will always depend on the latest generic kernel image 2024-04-22T05:38:47.726 INFO:teuthology.orchestra.run.smithi040.stdout: available. 2024-04-22T05:38:47.727 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-04-22T05:38:47.771 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.uZlgkGBKVp 2024-04-22T05:38:47.772 DEBUG:teuthology.orchestra.run.smithi040:> sudo cp /boot/grub/grub.cfg /tmp/tmp.uZlgkGBKVp 2024-04-22T05:38:47.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.uZlgkGBKVp 2024-04-22T05:38:47.884 DEBUG:teuthology.orchestra.run.smithi062:> dpkg -s linux-image-generic 2024-04-22T05:38:47.903 INFO:teuthology.orchestra.run.smithi062.stdout:Package: linux-image-generic 2024-04-22T05:38:47.903 INFO:teuthology.orchestra.run.smithi062.stdout:Status: install ok installed 2024-04-22T05:38:47.903 INFO:teuthology.orchestra.run.smithi062.stdout:Priority: optional 2024-04-22T05:38:47.903 INFO:teuthology.orchestra.run.smithi062.stdout:Section: kernel 2024-04-22T05:38:47.903 INFO:teuthology.orchestra.run.smithi062.stdout:Installed-Size: 21 2024-04-22T05:38:47.903 INFO:teuthology.orchestra.run.smithi062.stdout:Maintainer: Ubuntu Kernel Team 2024-04-22T05:38:47.903 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: amd64 2024-04-22T05:38:47.903 INFO:teuthology.orchestra.run.smithi062.stdout:Source: linux-meta 2024-04-22T05:38:47.904 INFO:teuthology.orchestra.run.smithi062.stdout:Version: 5.15.0.105.102 2024-04-22T05:38:47.904 INFO:teuthology.orchestra.run.smithi062.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-22T05:38:47.904 INFO:teuthology.orchestra.run.smithi062.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-22T05:38:47.904 INFO:teuthology.orchestra.run.smithi062.stdout:Recommends: thermald 2024-04-22T05:38:47.904 INFO:teuthology.orchestra.run.smithi062.stdout:Description: Generic Linux kernel image 2024-04-22T05:38:47.904 INFO:teuthology.orchestra.run.smithi062.stdout: This package will always depend on the latest generic kernel image 2024-04-22T05:38:47.904 INFO:teuthology.orchestra.run.smithi062.stdout: available. 2024-04-22T05:38:47.904 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-22T05:38:47.904 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-22T05:38:47.904 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-22T05:38:47.904 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-22T05:38:47.904 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi062.front.sepia.ceph.com, path=None, version=distro) 2024-04-22T05:38:47.905 DEBUG:teuthology.orchestra.run.smithi062:> sudo apt-get clean 2024-04-22T05:38:47.951 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.uZlgkGBKVp is 10KB 2024-04-22T05:38:47.963 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.uZlgkGBKVp 2024-04-22T05:38:47.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-22T05:38:48.023 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:38:48.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-22T05:38:48.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-22T05:38:48.096 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-22T05:38:48.096 DEBUG:teuthology.orchestra.run.smithi040:> sudo update-grub 2024-04-22T05:38:48.100 DEBUG:teuthology.orchestra.run.smithi062:> sudo apt-get update 2024-04-22T05:38:48.224 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-22T05:38:48.238 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-22T05:38:48.259 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-22T05:38:48.350 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-22T05:38:49.429 INFO:teuthology.orchestra.run.smithi040.stderr:Sourcing file `/etc/default/grub' 2024-04-22T05:38:49.430 INFO:teuthology.orchestra.run.smithi040.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-22T05:38:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:Generating grub configuration file ... 2024-04-22T05:38:49.651 INFO:teuthology.orchestra.run.smithi062.stdout:Reading package lists... 2024-04-22T05:38:49.669 DEBUG:teuthology.orchestra.run.smithi062:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-22T05:38:49.678 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-22T05:38:49.684 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-22T05:38:49.730 INFO:teuthology.orchestra.run.smithi062.stdout:Reading package lists... 2024-04-22T05:38:49.934 INFO:teuthology.orchestra.run.smithi062.stdout:Building dependency tree... 2024-04-22T05:38:49.934 INFO:teuthology.orchestra.run.smithi062.stdout:Reading state information... 2024-04-22T05:38:50.015 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-22T05:38:50.021 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-22T05:38:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-22T05:38:50.070 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-22T05:38:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-22T05:38:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:The following packages were automatically installed and are no longer required: 2024-04-22T05:38:50.118 INFO:teuthology.orchestra.run.smithi062.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-22T05:38:50.118 INFO:teuthology.orchestra.run.smithi062.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-22T05:38:50.118 INFO:teuthology.orchestra.run.smithi062.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-22T05:38:50.118 INFO:teuthology.orchestra.run.smithi062.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-22T05:38:50.119 INFO:teuthology.orchestra.run.smithi062.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-22T05:38:50.119 INFO:teuthology.orchestra.run.smithi062.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-22T05:38:50.161 INFO:teuthology.orchestra.run.smithi062.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-22T05:38:50.163 DEBUG:teuthology.orchestra.run.smithi062:> dpkg -s linux-image-generic 2024-04-22T05:38:50.176 INFO:teuthology.orchestra.run.smithi062.stdout:Package: linux-image-generic 2024-04-22T05:38:50.176 INFO:teuthology.orchestra.run.smithi062.stdout:Status: install ok installed 2024-04-22T05:38:50.176 INFO:teuthology.orchestra.run.smithi062.stdout:Priority: optional 2024-04-22T05:38:50.176 INFO:teuthology.orchestra.run.smithi062.stdout:Section: kernel 2024-04-22T05:38:50.176 INFO:teuthology.orchestra.run.smithi062.stdout:Installed-Size: 21 2024-04-22T05:38:50.176 INFO:teuthology.orchestra.run.smithi062.stdout:Maintainer: Ubuntu Kernel Team 2024-04-22T05:38:50.176 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: amd64 2024-04-22T05:38:50.176 INFO:teuthology.orchestra.run.smithi062.stdout:Source: linux-meta 2024-04-22T05:38:50.176 INFO:teuthology.orchestra.run.smithi062.stdout:Version: 5.15.0.105.102 2024-04-22T05:38:50.176 INFO:teuthology.orchestra.run.smithi062.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-22T05:38:50.177 INFO:teuthology.orchestra.run.smithi062.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-22T05:38:50.177 INFO:teuthology.orchestra.run.smithi062.stdout:Recommends: thermald 2024-04-22T05:38:50.177 INFO:teuthology.orchestra.run.smithi062.stdout:Description: Generic Linux kernel image 2024-04-22T05:38:50.177 INFO:teuthology.orchestra.run.smithi062.stdout: This package will always depend on the latest generic kernel image 2024-04-22T05:38:50.177 INFO:teuthology.orchestra.run.smithi062.stdout: available. 2024-04-22T05:38:50.177 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-04-22T05:38:50.223 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.DgImCuLeit 2024-04-22T05:38:50.223 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /boot/grub/grub.cfg /tmp/tmp.DgImCuLeit 2024-04-22T05:38:50.224 INFO:teuthology.orchestra.run.smithi040.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-22T05:38:50.225 INFO:teuthology.orchestra.run.smithi040.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-22T05:38:50.225 INFO:teuthology.orchestra.run.smithi040.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-22T05:38:50.253 INFO:teuthology.orchestra.run.smithi040.stderr:done 2024-04-22T05:38:50.262 DEBUG:teuthology.orchestra.run.smithi040:> sudo shutdown -r now 2024-04-22T05:38:50.279 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.DgImCuLeit 2024-04-22T05:38:50.392 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.DgImCuLeit is 10KB 2024-04-22T05:38:50.405 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.DgImCuLeit 2024-04-22T05:38:50.411 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-22T05:38:50.466 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:38:50.467 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-22T05:38:50.467 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-22T05:38:50.534 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-22T05:38:50.534 DEBUG:teuthology.orchestra.run.smithi062:> sudo update-grub 2024-04-22T05:38:51.666 INFO:teuthology.orchestra.run.smithi062.stderr:Sourcing file `/etc/default/grub' 2024-04-22T05:38:51.667 INFO:teuthology.orchestra.run.smithi062.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-22T05:38:51.669 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2024-04-22T05:38:51.853 INFO:teuthology.orchestra.run.smithi062.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-22T05:38:51.859 INFO:teuthology.orchestra.run.smithi062.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-22T05:38:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-22T05:38:52.091 INFO:teuthology.orchestra.run.smithi062.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-22T05:38:52.133 INFO:teuthology.orchestra.run.smithi062.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-22T05:38:52.139 INFO:teuthology.orchestra.run.smithi062.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-22T05:38:52.303 INFO:teuthology.orchestra.run.smithi062.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-22T05:38:52.303 INFO:teuthology.orchestra.run.smithi062.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-22T05:38:52.303 INFO:teuthology.orchestra.run.smithi062.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-22T05:38:52.322 INFO:teuthology.orchestra.run.smithi062.stderr:done 2024-04-22T05:38:52.325 DEBUG:teuthology.orchestra.run.smithi062:> sudo shutdown -r now 2024-04-22T05:39:20.293 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-22T05:39:20.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-22T05:39:20.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T05:39:22.329 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-22T05:39:22.329 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-22T05:39:22.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T05:39:56.831 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-04-22T05:40:05.839 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-22T05:40:05.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T05:40:20.310 DEBUG:teuthology.orchestra.remote:timed out 2024-04-22T05:40:24.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-04-22T05:40:29.317 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-22T05:40:29.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T05:40:36.230 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-22T05:40:36.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T05:40:36.765 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-04-22T05:40:36.847 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-04-22T05:40:37.484 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-22T05:40:37.484 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-22T05:40:37.484 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-04-22T05:40:37.531 INFO:teuthology.orchestra.run.smithi040.stdout:5.15.0-105-generic 2024-04-22T05:40:37.531 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-22T05:40:37.531 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-22T05:40:37.532 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-22T05:40:37.603 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-22T05:40:37.603 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-22T05:40:37.603 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-04-22T05:40:37.652 INFO:teuthology.orchestra.run.smithi062.stdout:5.15.0-105-generic 2024-04-22T05:40:37.653 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-22T05:40:37.653 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-22T05:40:37.653 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-22T05:40:38.533 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-22T05:40:38.533 DEBUG:teuthology.orchestra.run.smithi040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T05:40:38.612 INFO:teuthology.orchestra.run.smithi040.stdout:ttyS1 2024-04-22T05:40:38.636 DEBUG:teuthology.parallel:result is None 2024-04-22T05:40:38.654 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-22T05:40:38.654 DEBUG:teuthology.orchestra.run.smithi062:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T05:40:38.788 INFO:teuthology.orchestra.run.smithi062.stdout:ttyS1 2024-04-22T05:40:38.813 DEBUG:teuthology.parallel:result is None 2024-04-22T05:40:38.813 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-22T05:40:38.820 INFO:teuthology.task.internal:Creating test directory... 2024-04-22T05:40:38.820 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T05:40:38.822 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T05:40:38.827 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-22T05:40:38.834 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-22T05:40:38.841 INFO:teuthology.task.internal:Creating archive directory... 2024-04-22T05:40:38.841 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T05:40:38.868 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T05:40:38.892 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-22T05:40:38.899 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-22T05:40:38.899 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T05:40:38.932 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T05:40:38.946 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T05:40:38.952 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T05:40:38.957 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T05:40:38.963 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T05:40:38.965 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-22T05:40:38.972 INFO:teuthology.task.internal:Configuring sudo... 2024-04-22T05:40:38.973 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T05:40:39.000 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T05:40:39.017 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-22T05:40:39.026 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-22T05:40:39.026 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T05:40:39.056 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T05:40:39.063 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T05:40:39.107 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T05:40:39.155 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:40:39.155 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T05:40:39.238 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T05:40:39.243 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T05:40:39.291 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:40:39.291 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T05:40:39.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-04-22T05:40:39.368 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-04-22T05:40:39.513 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-22T05:40:39.521 INFO:teuthology.task.internal:Starting timer... 2024-04-22T05:40:39.521 INFO:teuthology.run_tasks:Running task pcp... 2024-04-22T05:40:39.531 INFO:teuthology.run_tasks:Running task selinux... 2024-04-22T05:40:39.540 DEBUG:teuthology.task.selinux:Excluding smithi040: OS 'ubuntu' does not support SELinux 2024-04-22T05:40:39.540 DEBUG:teuthology.task.selinux:Excluding smithi062: OS 'ubuntu' does not support SELinux 2024-04-22T05:40:39.540 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-22T05:40:39.540 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-22T05:40:39.541 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-22T05:40:39.541 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-22T05:40:39.552 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-22T05:40:39.558 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-22T05:40:39.707 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-22T05:40:39.723 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-22T05:40:39.726 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi062.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-22T05:47:44.939 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi062.front.sepia.ceph.com')] 2024-04-22T05:47:44.940 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-22T05:47:44.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T05:47:45.037 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-04-22T05:47:45.114 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-22T05:47:45.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-22T05:47:45.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T05:47:45.201 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-04-22T05:47:45.282 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-22T05:47:45.283 INFO:teuthology.run_tasks:Running task clock... 2024-04-22T05:47:45.293 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-22T05:47:45.293 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T05:47:45.294 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T05:47:45.296 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T05:47:45.296 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T05:47:45.322 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-22T05:47:45.322 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: Command line: ntpd -gq 2024-04-22T05:47:45.322 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: ---------------------------------------------------- 2024-04-22T05:47:45.322 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: ntp-4 is maintained by Network Time Foundation, 2024-04-22T05:47:45.322 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-22T05:47:45.322 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: corporation. Support and training for ntp-4 are 2024-04-22T05:47:45.322 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: available at https://www.nwtime.org/support 2024-04-22T05:47:45.322 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: ---------------------------------------------------- 2024-04-22T05:47:45.323 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: proto: precision = 0.047 usec (-24) 2024-04-22T05:47:45.323 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: basedate set to 2022-02-04 2024-04-22T05:47:45.323 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: gps base set to 2022-02-06 (week 2196) 2024-04-22T05:47:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-22T05:47:45.324 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-22T05:47:45.324 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: restrict ::: KOD does nothing without LIMITED. 2024-04-22T05:47:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-22T05:47:45.325 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: Listen and drop on 0 v6wildcard [::]:123 2024-04-22T05:47:45.325 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-22T05:47:45.325 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: Listen normally on 2 lo 127.0.0.1:123 2024-04-22T05:47:45.325 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: Listen normally on 3 ens1f0 172.21.15.40:123 2024-04-22T05:47:45.325 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: Listen normally on 4 lo [::1]:123 2024-04-22T05:47:45.325 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:12cc%4]:123 2024-04-22T05:47:45.325 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:45 ntpd[17106]: Listening on routing socket on fd #22 for interface updates 2024-04-22T05:47:45.353 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-22T05:47:45.353 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: Command line: ntpd -gq 2024-04-22T05:47:45.353 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: ---------------------------------------------------- 2024-04-22T05:47:45.353 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: ntp-4 is maintained by Network Time Foundation, 2024-04-22T05:47:45.353 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-22T05:47:45.353 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: corporation. Support and training for ntp-4 are 2024-04-22T05:47:45.353 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: available at https://www.nwtime.org/support 2024-04-22T05:47:45.353 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: ---------------------------------------------------- 2024-04-22T05:47:45.354 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: proto: precision = 0.046 usec (-24) 2024-04-22T05:47:45.354 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: basedate set to 2022-02-04 2024-04-22T05:47:45.354 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: gps base set to 2022-02-06 (week 2196) 2024-04-22T05:47:45.354 INFO:teuthology.orchestra.run.smithi062.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-22T05:47:45.355 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-22T05:47:45.355 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: restrict ::: KOD does nothing without LIMITED. 2024-04-22T05:47:45.355 INFO:teuthology.orchestra.run.smithi062.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-22T05:47:45.356 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: Listen and drop on 0 v6wildcard [::]:123 2024-04-22T05:47:45.356 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-22T05:47:45.356 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: Listen normally on 2 lo 127.0.0.1:123 2024-04-22T05:47:45.356 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: Listen normally on 3 enp3s0f1 172.21.15.62:123 2024-04-22T05:47:45.356 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: Listen normally on 4 lo [::1]:123 2024-04-22T05:47:45.356 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:febb:d1c3%5]:123 2024-04-22T05:47:45.356 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:45 ntpd[17231]: Listening on routing socket on fd #22 for interface updates 2024-04-22T05:47:46.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:46 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:46.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:46 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:46.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:46 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:46.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:46 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:46.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:46 ntpd[17106]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:46.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:46 ntpd[17106]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:46.353 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:46 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:46.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:46 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:46.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:46 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:46.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:46 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:46.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:46 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:46.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:46 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:46.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:46 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:46.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:46 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:47.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:47 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:47.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:47 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:47.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:47 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:47.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:47 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:47.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:47 ntpd[17106]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:47.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:47 ntpd[17106]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:48.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:48 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:48.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:48 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:48.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:48 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:48.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:48 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:48.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:48 ntpd[17106]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:48.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:48 ntpd[17106]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:48.353 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:48 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:48.353 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:48 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:48.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:48 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:48.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:48 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:48.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:48 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:48.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:48 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:49.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:49 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:49.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:49 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:49.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:49 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:49 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:49 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:49 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:49 ntpd[17106]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:49 ntpd[17106]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:49 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:49 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:49 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:49 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:49.355 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:49 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:49.355 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:49 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:50.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:50 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:50.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:50 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:50.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:50 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:50.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:50 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:50.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:50 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:50.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:50 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:50.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:50 ntpd[17106]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:50.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:50 ntpd[17106]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:50.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:50 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:50.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:50 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:50.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:50 ntpd[17106]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:50.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:50 ntpd[17106]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:50.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:50 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:50.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:50 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:50.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:50 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:50.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:50 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:50.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:50 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:50.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:50 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:50.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:50 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:50.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:50 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:50.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:50 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:50.355 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:50 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:50.355 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:50 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:50.355 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:50 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:51 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:51 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:51 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:51 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:51 ntpd[17106]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:51 ntpd[17106]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:51.353 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:51 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:51.353 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:51 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:51.353 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:51 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:51.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:51 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:51.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:51 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:51.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:51 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:52 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:52 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:52 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:52 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:52 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:52 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:52 ntpd[17106]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:52 ntpd[17106]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:52 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:52 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:52 ntpd[17106]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:52.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:52 ntpd[17106]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:52.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:52 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:52.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:52 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:52.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:52 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:52.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:52 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:52.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:52 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:52.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:52 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:52.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:52 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:52.354 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:52 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:52.355 INFO:teuthology.orchestra.run.smithi062.stdout:22 Apr 05:47:52 ntpd[17231]: ntpd: time slew -0.000167 s 2024-04-22T05:47:52.355 INFO:teuthology.orchestra.run.smithi062.stdout:ntpd: time slew -0.000167s 2024-04-22T05:47:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:52 ntpd[17231]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-22T05:47:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:22 Apr 05:47:52 ntpd[17231]: can't open /var/log/ntpstats/loopstats.20240422: Permission denied 2024-04-22T05:47:52.410 INFO:teuthology.orchestra.run.smithi062.stdout: remote refid st t when poll reach delay offset jitter 2024-04-22T05:47:52.411 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================== 2024-04-22T05:47:52.411 INFO:teuthology.orchestra.run.smithi062.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-22T05:47:52.411 INFO:teuthology.orchestra.run.smithi062.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-22T05:47:52.411 INFO:teuthology.orchestra.run.smithi062.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-22T05:47:52.411 INFO:teuthology.orchestra.run.smithi062.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-22T05:47:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:53 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:53 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:53 ntpd[17106]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-22T05:47:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:53 ntpd[17106]: can't open /var/log/ntpstats/rawstats.20240422: Permission denied 2024-04-22T05:47:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:53 ntpd[17106]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-22T05:47:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:53 ntpd[17106]: can't open /var/log/ntpstats/peerstats.20240422: Permission denied 2024-04-22T05:47:53.324 INFO:teuthology.orchestra.run.smithi040.stdout:22 Apr 05:47:53 ntpd[17106]: ntpd: time slew -0.000839 s 2024-04-22T05:47:53.324 INFO:teuthology.orchestra.run.smithi040.stdout:ntpd: time slew -0.000839s 2024-04-22T05:47:53.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:53 ntpd[17106]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-22T05:47:53.324 INFO:teuthology.orchestra.run.smithi040.stderr:22 Apr 05:47:53 ntpd[17106]: can't open /var/log/ntpstats/loopstats.20240422: Permission denied 2024-04-22T05:47:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: remote refid st t when poll reach delay offset jitter 2024-04-22T05:47:53.376 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================== 2024-04-22T05:47:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-22T05:47:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-22T05:47:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-22T05:47:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-22T05:47:53.377 INFO:teuthology.run_tasks:Running task install... 2024-04-22T05:47:53.387 DEBUG:teuthology.task.install:project ceph 2024-04-22T05:47:53.387 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'}} 2024-04-22T05:47:53.387 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'} 2024-04-22T05:47:53.387 INFO:teuthology.task.install:Using flavor: default 2024-04-22T05:47:53.394 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-22T05:47:53.394 INFO:teuthology.task.install:extra packages: [] 2024-04-22T05:47:53.395 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-key list | grep Ceph 2024-04-22T05:47:53.395 DEBUG:teuthology.orchestra.run.smithi062:> sudo apt-key list | grep Ceph 2024-04-22T05:47:53.462 INFO:teuthology.orchestra.run.smithi062.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-22T05:47:53.474 INFO:teuthology.orchestra.run.smithi040.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-22T05:47:53.492 INFO:teuthology.orchestra.run.smithi062.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-22T05:47:53.492 INFO:teuthology.orchestra.run.smithi062.stdout:uid [ unknown] Ceph.com (release key) 2024-04-22T05:47:53.493 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-22T05:47:53.493 INFO:teuthology.task.install.deb:Installing system (non-project) packages: sqlite3 on remote deb x86_64 2024-04-22T05:47:53.493 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-22T05:47:53.502 INFO:teuthology.orchestra.run.smithi040.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-22T05:47:53.502 INFO:teuthology.orchestra.run.smithi040.stdout:uid [ unknown] Ceph.com (release key) 2024-04-22T05:47:53.503 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-22T05:47:53.503 INFO:teuthology.task.install.deb:Installing system (non-project) packages: sqlite3 on remote deb x86_64 2024-04-22T05:47:53.503 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-22T05:47:53.636 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default/ 2024-04-22T05:47:53.636 INFO:teuthology.task.install.deb:Package version is 18.2.2-1237-g646b8c7d-1jammy 2024-04-22T05:47:53.646 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default/ 2024-04-22T05:47:53.646 INFO:teuthology.task.install.deb:Package version is 18.2.2-1237-g646b8c7d-1jammy 2024-04-22T05:47:53.747 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:47:53.747 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-22T05:47:53.762 DEBUG:teuthology.orchestra.run.smithi062:> sudo apt-get update 2024-04-22T05:47:53.816 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:47:53.816 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-22T05:47:53.830 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get update 2024-04-22T05:47:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-22T05:47:53.941 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-22T05:47:53.955 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-22T05:47:53.976 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-22T05:47:54.006 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-22T05:47:54.006 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-22T05:47:54.020 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-22T05:47:54.040 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-22T05:47:54.050 INFO:teuthology.orchestra.run.smithi062.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy InRelease 2024-04-22T05:47:54.087 INFO:teuthology.orchestra.run.smithi062.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-22T05:47:54.113 INFO:teuthology.orchestra.run.smithi040.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy InRelease 2024-04-22T05:47:54.124 INFO:teuthology.orchestra.run.smithi062.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-22T05:47:54.149 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-22T05:47:54.161 INFO:teuthology.orchestra.run.smithi062.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,429 B] 2024-04-22T05:47:54.186 INFO:teuthology.orchestra.run.smithi040.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-22T05:47:54.206 INFO:teuthology.orchestra.run.smithi062.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2024-04-22T05:47:54.222 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,429 B] 2024-04-22T05:47:54.266 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2024-04-22T05:47:54.309 INFO:teuthology.orchestra.run.smithi062.stdout:Fetched 28.6 kB in 0s (78.1 kB/s) 2024-04-22T05:47:54.365 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 28.6 kB in 0s (80.3 kB/s) 2024-04-22T05:47:55.283 INFO:teuthology.orchestra.run.smithi062.stdout:Reading package lists... 2024-04-22T05:47:55.299 DEBUG:teuthology.orchestra.run.smithi062:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1237-g646b8c7d-1jammy cephadm=18.2.2-1237-g646b8c7d-1jammy ceph-mds=18.2.2-1237-g646b8c7d-1jammy ceph-mgr=18.2.2-1237-g646b8c7d-1jammy ceph-common=18.2.2-1237-g646b8c7d-1jammy ceph-fuse=18.2.2-1237-g646b8c7d-1jammy ceph-test=18.2.2-1237-g646b8c7d-1jammy ceph-volume=18.2.2-1237-g646b8c7d-1jammy radosgw=18.2.2-1237-g646b8c7d-1jammy python3-rados=18.2.2-1237-g646b8c7d-1jammy python3-rgw=18.2.2-1237-g646b8c7d-1jammy python3-cephfs=18.2.2-1237-g646b8c7d-1jammy python3-rbd=18.2.2-1237-g646b8c7d-1jammy libcephfs2=18.2.2-1237-g646b8c7d-1jammy libcephfs-dev=18.2.2-1237-g646b8c7d-1jammy librados2=18.2.2-1237-g646b8c7d-1jammy librbd1=18.2.2-1237-g646b8c7d-1jammy rbd-fuse=18.2.2-1237-g646b8c7d-1jammy 2024-04-22T05:47:55.327 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-22T05:47:55.344 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1237-g646b8c7d-1jammy cephadm=18.2.2-1237-g646b8c7d-1jammy ceph-mds=18.2.2-1237-g646b8c7d-1jammy ceph-mgr=18.2.2-1237-g646b8c7d-1jammy ceph-common=18.2.2-1237-g646b8c7d-1jammy ceph-fuse=18.2.2-1237-g646b8c7d-1jammy ceph-test=18.2.2-1237-g646b8c7d-1jammy ceph-volume=18.2.2-1237-g646b8c7d-1jammy radosgw=18.2.2-1237-g646b8c7d-1jammy python3-rados=18.2.2-1237-g646b8c7d-1jammy python3-rgw=18.2.2-1237-g646b8c7d-1jammy python3-cephfs=18.2.2-1237-g646b8c7d-1jammy python3-rbd=18.2.2-1237-g646b8c7d-1jammy libcephfs2=18.2.2-1237-g646b8c7d-1jammy libcephfs-dev=18.2.2-1237-g646b8c7d-1jammy librados2=18.2.2-1237-g646b8c7d-1jammy librbd1=18.2.2-1237-g646b8c7d-1jammy rbd-fuse=18.2.2-1237-g646b8c7d-1jammy 2024-04-22T05:47:55.361 INFO:teuthology.orchestra.run.smithi062.stdout:Reading package lists... 2024-04-22T05:47:55.409 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-22T05:47:55.564 INFO:teuthology.orchestra.run.smithi062.stdout:Building dependency tree... 2024-04-22T05:47:55.564 INFO:teuthology.orchestra.run.smithi062.stdout:Reading state information... 2024-04-22T05:47:55.611 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2024-04-22T05:47:55.612 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2024-04-22T05:47:55.764 INFO:teuthology.orchestra.run.smithi062.stdout:The following packages were automatically installed and are no longer required: 2024-04-22T05:47:55.764 INFO:teuthology.orchestra.run.smithi062.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-22T05:47:55.764 INFO:teuthology.orchestra.run.smithi062.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-22T05:47:55.764 INFO:teuthology.orchestra.run.smithi062.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-22T05:47:55.765 INFO:teuthology.orchestra.run.smithi062.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-22T05:47:55.765 INFO:teuthology.orchestra.run.smithi062.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-22T05:47:55.765 INFO:teuthology.orchestra.run.smithi062.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-22T05:47:55.766 INFO:teuthology.orchestra.run.smithi062.stdout:The following additional packages will be installed: 2024-04-22T05:47:55.766 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-22T05:47:55.766 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-22T05:47:55.766 INFO:teuthology.orchestra.run.smithi062.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-04-22T05:47:55.766 INFO:teuthology.orchestra.run.smithi062.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-22T05:47:55.767 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-04-22T05:47:55.767 INFO:teuthology.orchestra.run.smithi062.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-22T05:47:55.767 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-22T05:47:55.767 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-22T05:47:55.767 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-22T05:47:55.767 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-22T05:47:55.767 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-22T05:47:55.767 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-22T05:47:55.767 INFO:teuthology.orchestra.run.smithi062.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-22T05:47:55.767 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-22T05:47:55.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-22T05:47:55.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-22T05:47:55.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-22T05:47:55.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-22T05:47:55.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-22T05:47:55.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-22T05:47:55.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-22T05:47:55.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-04-22T05:47:55.768 INFO:teuthology.orchestra.run.smithi062.stdout:Suggested packages: 2024-04-22T05:47:55.768 INFO:teuthology.orchestra.run.smithi062.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-04-22T05:47:55.769 INFO:teuthology.orchestra.run.smithi062.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-04-22T05:47:55.769 INFO:teuthology.orchestra.run.smithi062.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-04-22T05:47:55.769 INFO:teuthology.orchestra.run.smithi062.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-04-22T05:47:55.769 INFO:teuthology.orchestra.run.smithi062.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-04-22T05:47:55.769 INFO:teuthology.orchestra.run.smithi062.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-04-22T05:47:55.769 INFO:teuthology.orchestra.run.smithi062.stdout: python3-watchdog 2024-04-22T05:47:55.769 INFO:teuthology.orchestra.run.smithi062.stdout:Recommended packages: 2024-04-22T05:47:55.769 INFO:teuthology.orchestra.run.smithi062.stdout: btrfs-tools 2024-04-22T05:47:55.813 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2024-04-22T05:47:55.813 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-22T05:47:55.814 INFO:teuthology.orchestra.run.smithi040.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-22T05:47:55.814 INFO:teuthology.orchestra.run.smithi040.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-22T05:47:55.814 INFO:teuthology.orchestra.run.smithi040.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-22T05:47:55.814 INFO:teuthology.orchestra.run.smithi040.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-22T05:47:55.814 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-22T05:47:55.815 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2024-04-22T05:47:55.815 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-22T05:47:55.816 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-22T05:47:55.816 INFO:teuthology.orchestra.run.smithi040.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-04-22T05:47:55.816 INFO:teuthology.orchestra.run.smithi040.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-22T05:47:55.816 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-04-22T05:47:55.816 INFO:teuthology.orchestra.run.smithi040.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-22T05:47:55.816 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-22T05:47:55.816 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-22T05:47:55.816 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-22T05:47:55.816 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-22T05:47:55.816 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-22T05:47:55.817 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-22T05:47:55.817 INFO:teuthology.orchestra.run.smithi040.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-22T05:47:55.817 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-22T05:47:55.817 INFO:teuthology.orchestra.run.smithi040.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-22T05:47:55.817 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-22T05:47:55.817 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-22T05:47:55.817 INFO:teuthology.orchestra.run.smithi040.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-22T05:47:55.817 INFO:teuthology.orchestra.run.smithi040.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-22T05:47:55.818 INFO:teuthology.orchestra.run.smithi040.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-22T05:47:55.818 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-22T05:47:55.818 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-04-22T05:47:55.819 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2024-04-22T05:47:55.819 INFO:teuthology.orchestra.run.smithi040.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-04-22T05:47:55.819 INFO:teuthology.orchestra.run.smithi040.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-04-22T05:47:55.819 INFO:teuthology.orchestra.run.smithi040.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-04-22T05:47:55.819 INFO:teuthology.orchestra.run.smithi040.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-04-22T05:47:55.819 INFO:teuthology.orchestra.run.smithi040.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-04-22T05:47:55.820 INFO:teuthology.orchestra.run.smithi040.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-04-22T05:47:55.820 INFO:teuthology.orchestra.run.smithi040.stdout: python3-watchdog 2024-04-22T05:47:55.820 INFO:teuthology.orchestra.run.smithi040.stdout:Recommended packages: 2024-04-22T05:47:55.820 INFO:teuthology.orchestra.run.smithi040.stdout: btrfs-tools 2024-04-22T05:47:55.872 INFO:teuthology.orchestra.run.smithi062.stdout:The following NEW packages will be installed: 2024-04-22T05:47:55.872 INFO:teuthology.orchestra.run.smithi062.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-22T05:47:55.872 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-22T05:47:55.873 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-22T05:47:55.874 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-04-22T05:47:55.874 INFO:teuthology.orchestra.run.smithi062.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-04-22T05:47:55.874 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-04-22T05:47:55.874 INFO:teuthology.orchestra.run.smithi062.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-04-22T05:47:55.874 INFO:teuthology.orchestra.run.smithi062.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-22T05:47:55.874 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-22T05:47:55.874 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-04-22T05:47:55.874 INFO:teuthology.orchestra.run.smithi062.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-22T05:47:55.874 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-22T05:47:55.874 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-22T05:47:55.874 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-22T05:47:55.875 INFO:teuthology.orchestra.run.smithi062.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-22T05:47:55.875 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-22T05:47:55.875 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-22T05:47:55.875 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-22T05:47:55.875 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-22T05:47:55.875 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-22T05:47:55.875 INFO:teuthology.orchestra.run.smithi062.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-22T05:47:55.875 INFO:teuthology.orchestra.run.smithi062.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-22T05:47:55.875 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-22T05:47:55.875 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-22T05:47:55.875 INFO:teuthology.orchestra.run.smithi062.stdout: socat xmlstarlet 2024-04-22T05:47:55.921 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2024-04-22T05:47:55.921 INFO:teuthology.orchestra.run.smithi040.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-22T05:47:55.921 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-22T05:47:55.921 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-22T05:47:55.921 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-04-22T05:47:55.921 INFO:teuthology.orchestra.run.smithi040.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-04-22T05:47:55.921 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-04-22T05:47:55.922 INFO:teuthology.orchestra.run.smithi040.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-04-22T05:47:55.922 INFO:teuthology.orchestra.run.smithi040.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-22T05:47:55.922 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-22T05:47:55.922 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-04-22T05:47:55.922 INFO:teuthology.orchestra.run.smithi040.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-22T05:47:55.922 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-22T05:47:55.922 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-22T05:47:55.923 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-22T05:47:55.923 INFO:teuthology.orchestra.run.smithi040.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-22T05:47:55.923 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-22T05:47:55.923 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-22T05:47:55.923 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-22T05:47:55.923 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-22T05:47:55.923 INFO:teuthology.orchestra.run.smithi040.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-22T05:47:55.923 INFO:teuthology.orchestra.run.smithi040.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-22T05:47:55.923 INFO:teuthology.orchestra.run.smithi040.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-22T05:47:55.923 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-22T05:47:55.923 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-22T05:47:55.923 INFO:teuthology.orchestra.run.smithi040.stdout: socat xmlstarlet 2024-04-22T05:47:55.938 INFO:teuthology.orchestra.run.smithi062.stdout:0 upgraded, 109 newly installed, 0 to remove and 302 not upgraded. 2024-04-22T05:47:55.938 INFO:teuthology.orchestra.run.smithi062.stdout:Need to get 162 MB of archives. 2024-04-22T05:47:55.938 INFO:teuthology.orchestra.run.smithi062.stdout:After this operation, 743 MB of additional disk space will be used. 2024-04-22T05:47:55.938 INFO:teuthology.orchestra.run.smithi062.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-04-22T05:47:55.991 INFO:teuthology.orchestra.run.smithi040.stdout:0 upgraded, 109 newly installed, 0 to remove and 302 not upgraded. 2024-04-22T05:47:55.991 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 162 MB of archives. 2024-04-22T05:47:55.991 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 743 MB of additional disk space will be used. 2024-04-22T05:47:55.991 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-04-22T05:47:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-22T05:47:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-22T05:47:56.048 INFO:teuthology.orchestra.run.smithi062.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-22T05:47:56.060 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-22T05:47:56.065 INFO:teuthology.orchestra.run.smithi062.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-22T05:47:56.069 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.2-1237-g646b8c7d-1jammy [3,440 kB] 2024-04-22T05:47:56.092 INFO:teuthology.orchestra.run.smithi062.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.2-1237-g646b8c7d-1jammy [3,440 kB] 2024-04-22T05:47:56.104 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-22T05:47:56.107 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-22T05:47:56.121 INFO:teuthology.orchestra.run.smithi062.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-22T05:47:56.125 INFO:teuthology.orchestra.run.smithi062.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-22T05:47:56.129 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-22T05:47:56.138 INFO:teuthology.orchestra.run.smithi062.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-22T05:47:56.142 INFO:teuthology.orchestra.run.smithi062.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-22T05:47:56.142 INFO:teuthology.orchestra.run.smithi062.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-22T05:47:56.143 INFO:teuthology.orchestra.run.smithi062.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-22T05:47:56.152 INFO:teuthology.orchestra.run.smithi062.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-22T05:47:56.152 INFO:teuthology.orchestra.run.smithi062.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-22T05:47:56.153 INFO:teuthology.orchestra.run.smithi062.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-22T05:47:56.154 INFO:teuthology.orchestra.run.smithi062.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-22T05:47:56.164 INFO:teuthology.orchestra.run.smithi062.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-22T05:47:56.165 INFO:teuthology.orchestra.run.smithi062.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-22T05:47:56.165 INFO:teuthology.orchestra.run.smithi062.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-22T05:47:56.185 INFO:teuthology.orchestra.run.smithi062.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-22T05:47:56.186 INFO:teuthology.orchestra.run.smithi062.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-22T05:47:56.187 INFO:teuthology.orchestra.run.smithi062.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-22T05:47:56.189 INFO:teuthology.orchestra.run.smithi062.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-22T05:47:56.190 INFO:teuthology.orchestra.run.smithi062.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-22T05:47:56.191 INFO:teuthology.orchestra.run.smithi062.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-22T05:47:56.191 INFO:teuthology.orchestra.run.smithi062.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-22T05:47:56.192 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-22T05:47:56.197 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-22T05:47:56.210 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-22T05:47:56.214 INFO:teuthology.orchestra.run.smithi040.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-22T05:47:56.218 INFO:teuthology.orchestra.run.smithi040.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-22T05:47:56.218 INFO:teuthology.orchestra.run.smithi040.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-22T05:47:56.225 INFO:teuthology.orchestra.run.smithi040.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-22T05:47:56.225 INFO:teuthology.orchestra.run.smithi040.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-22T05:47:56.227 INFO:teuthology.orchestra.run.smithi040.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-22T05:47:56.229 INFO:teuthology.orchestra.run.smithi062.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-22T05:47:56.229 INFO:teuthology.orchestra.run.smithi062.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-22T05:47:56.230 INFO:teuthology.orchestra.run.smithi062.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-22T05:47:56.240 INFO:teuthology.orchestra.run.smithi040.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-22T05:47:56.255 INFO:teuthology.orchestra.run.smithi040.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-22T05:47:56.256 INFO:teuthology.orchestra.run.smithi062.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-22T05:47:56.256 INFO:teuthology.orchestra.run.smithi040.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-22T05:47:56.256 INFO:teuthology.orchestra.run.smithi040.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-22T05:47:56.257 INFO:teuthology.orchestra.run.smithi062.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-22T05:47:56.257 INFO:teuthology.orchestra.run.smithi062.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-22T05:47:56.257 INFO:teuthology.orchestra.run.smithi040.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-22T05:47:56.258 INFO:teuthology.orchestra.run.smithi040.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-22T05:47:56.258 INFO:teuthology.orchestra.run.smithi040.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-22T05:47:56.261 INFO:teuthology.orchestra.run.smithi040.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-22T05:47:56.262 INFO:teuthology.orchestra.run.smithi040.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-22T05:47:56.264 INFO:teuthology.orchestra.run.smithi062.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-22T05:47:56.264 INFO:teuthology.orchestra.run.smithi062.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-22T05:47:56.264 INFO:teuthology.orchestra.run.smithi062.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-22T05:47:56.265 INFO:teuthology.orchestra.run.smithi062.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-22T05:47:56.276 INFO:teuthology.orchestra.run.smithi040.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-22T05:47:56.286 INFO:teuthology.orchestra.run.smithi040.stdout:Get:26 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.2-1237-g646b8c7d-1jammy [3,106 kB] 2024-04-22T05:47:56.288 INFO:teuthology.orchestra.run.smithi062.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-22T05:47:56.290 INFO:teuthology.orchestra.run.smithi062.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-22T05:47:56.291 INFO:teuthology.orchestra.run.smithi062.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-22T05:47:56.294 INFO:teuthology.orchestra.run.smithi040.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-22T05:47:56.296 INFO:teuthology.orchestra.run.smithi040.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-22T05:47:56.296 INFO:teuthology.orchestra.run.smithi040.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-22T05:47:56.297 INFO:teuthology.orchestra.run.smithi040.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-22T05:47:56.298 INFO:teuthology.orchestra.run.smithi040.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-22T05:47:56.299 INFO:teuthology.orchestra.run.smithi040.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-22T05:47:56.299 INFO:teuthology.orchestra.run.smithi040.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-22T05:47:56.306 INFO:teuthology.orchestra.run.smithi040.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-22T05:47:56.311 INFO:teuthology.orchestra.run.smithi040.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-22T05:47:56.324 INFO:teuthology.orchestra.run.smithi062.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-22T05:47:56.325 INFO:teuthology.orchestra.run.smithi062.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-22T05:47:56.327 INFO:teuthology.orchestra.run.smithi062.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-22T05:47:56.328 INFO:teuthology.orchestra.run.smithi062.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-22T05:47:56.331 INFO:teuthology.orchestra.run.smithi062.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-22T05:47:56.361 INFO:teuthology.orchestra.run.smithi040.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-22T05:47:56.363 INFO:teuthology.orchestra.run.smithi040.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-22T05:47:56.366 INFO:teuthology.orchestra.run.smithi040.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-22T05:47:56.370 INFO:teuthology.orchestra.run.smithi040.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-22T05:47:56.372 INFO:teuthology.orchestra.run.smithi040.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-22T05:47:56.375 INFO:teuthology.orchestra.run.smithi040.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-22T05:47:56.376 INFO:teuthology.orchestra.run.smithi040.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-22T05:47:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-22T05:47:56.382 INFO:teuthology.orchestra.run.smithi040.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-22T05:47:56.387 INFO:teuthology.orchestra.run.smithi040.stdout:Get:45 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.2-1237-g646b8c7d-1jammy [680 kB] 2024-04-22T05:47:56.410 INFO:teuthology.orchestra.run.smithi040.stdout:Get:46 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.2-1237-g646b8c7d-1jammy [356 kB] 2024-04-22T05:47:56.420 INFO:teuthology.orchestra.run.smithi040.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-22T05:47:56.421 INFO:teuthology.orchestra.run.smithi040.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.2-1237-g646b8c7d-1jammy [32.8 kB] 2024-04-22T05:47:56.421 INFO:teuthology.orchestra.run.smithi062.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-22T05:47:56.422 INFO:teuthology.orchestra.run.smithi040.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.2-1237-g646b8c7d-1jammy [180 kB] 2024-04-22T05:47:56.423 INFO:teuthology.orchestra.run.smithi062.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-22T05:47:56.425 INFO:teuthology.orchestra.run.smithi062.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-22T05:47:56.426 INFO:teuthology.orchestra.run.smithi062.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-22T05:47:56.426 INFO:teuthology.orchestra.run.smithi062.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-22T05:47:56.428 INFO:teuthology.orchestra.run.smithi040.stdout:Get:50 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.2-1237-g646b8c7d-1jammy [63.7 kB] 2024-04-22T05:47:56.430 INFO:teuthology.orchestra.run.smithi040.stdout:Get:51 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.2-1237-g646b8c7d-1jammy [330 kB] 2024-04-22T05:47:56.445 INFO:teuthology.orchestra.run.smithi040.stdout:Get:52 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.2-1237-g646b8c7d-1jammy [5,891 kB] 2024-04-22T05:47:56.448 INFO:teuthology.orchestra.run.smithi062.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-22T05:47:56.449 INFO:teuthology.orchestra.run.smithi062.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-22T05:47:56.533 INFO:teuthology.orchestra.run.smithi062.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-22T05:47:56.538 INFO:teuthology.orchestra.run.smithi062.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-22T05:47:56.538 INFO:teuthology.orchestra.run.smithi062.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-22T05:47:56.540 INFO:teuthology.orchestra.run.smithi062.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-22T05:47:56.558 INFO:teuthology.orchestra.run.smithi040.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-22T05:47:56.559 INFO:teuthology.orchestra.run.smithi040.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-22T05:47:56.561 INFO:teuthology.orchestra.run.smithi040.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-22T05:47:56.562 INFO:teuthology.orchestra.run.smithi040.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-22T05:47:56.563 INFO:teuthology.orchestra.run.smithi040.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-22T05:47:56.564 INFO:teuthology.orchestra.run.smithi040.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-22T05:47:56.565 INFO:teuthology.orchestra.run.smithi040.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-22T05:47:56.565 INFO:teuthology.orchestra.run.smithi040.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-22T05:47:56.569 INFO:teuthology.orchestra.run.smithi040.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-22T05:47:56.577 INFO:teuthology.orchestra.run.smithi062.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-22T05:47:56.581 INFO:teuthology.orchestra.run.smithi062.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-22T05:47:56.582 INFO:teuthology.orchestra.run.smithi040.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-22T05:47:56.582 INFO:teuthology.orchestra.run.smithi062.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-22T05:47:56.593 INFO:teuthology.orchestra.run.smithi040.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-22T05:47:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-22T05:47:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-22T05:47:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-22T05:47:56.609 INFO:teuthology.orchestra.run.smithi062.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-22T05:47:56.609 INFO:teuthology.orchestra.run.smithi062.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-22T05:47:56.610 INFO:teuthology.orchestra.run.smithi062.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-22T05:47:56.612 INFO:teuthology.orchestra.run.smithi062.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-22T05:47:56.627 INFO:teuthology.orchestra.run.smithi040.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-22T05:47:56.631 INFO:teuthology.orchestra.run.smithi040.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-22T05:47:56.632 INFO:teuthology.orchestra.run.smithi040.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-22T05:47:56.646 INFO:teuthology.orchestra.run.smithi062.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-22T05:47:56.652 INFO:teuthology.orchestra.run.smithi062.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-22T05:47:56.654 INFO:teuthology.orchestra.run.smithi040.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-22T05:47:56.655 INFO:teuthology.orchestra.run.smithi040.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-22T05:47:56.655 INFO:teuthology.orchestra.run.smithi062.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-22T05:47:56.655 INFO:teuthology.orchestra.run.smithi040.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-22T05:47:56.656 INFO:teuthology.orchestra.run.smithi040.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-22T05:47:56.656 INFO:teuthology.orchestra.run.smithi040.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-22T05:47:56.657 INFO:teuthology.orchestra.run.smithi062.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-22T05:47:56.659 INFO:teuthology.orchestra.run.smithi062.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-22T05:47:56.663 INFO:teuthology.orchestra.run.smithi040.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.2-1237-g646b8c7d-1jammy [112 kB] 2024-04-22T05:47:56.664 INFO:teuthology.orchestra.run.smithi062.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-22T05:47:56.665 INFO:teuthology.orchestra.run.smithi040.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-22T05:47:56.666 INFO:teuthology.orchestra.run.smithi040.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.2-1237-g646b8c7d-1jammy [441 kB] 2024-04-22T05:47:56.669 INFO:teuthology.orchestra.run.smithi062.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-22T05:47:56.677 INFO:teuthology.orchestra.run.smithi062.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-22T05:47:56.678 INFO:teuthology.orchestra.run.smithi040.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-22T05:47:56.679 INFO:teuthology.orchestra.run.smithi040.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-22T05:47:56.680 INFO:teuthology.orchestra.run.smithi040.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.2-1237-g646b8c7d-1jammy [21.9 MB] 2024-04-22T05:47:56.682 INFO:teuthology.orchestra.run.smithi062.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-22T05:47:56.683 INFO:teuthology.orchestra.run.smithi062.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-22T05:47:56.683 INFO:teuthology.orchestra.run.smithi040.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-22T05:47:56.685 INFO:teuthology.orchestra.run.smithi040.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-22T05:47:56.687 INFO:teuthology.orchestra.run.smithi040.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-22T05:47:56.688 INFO:teuthology.orchestra.run.smithi040.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-22T05:47:56.692 INFO:teuthology.orchestra.run.smithi040.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-22T05:47:56.694 INFO:teuthology.orchestra.run.smithi062.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-22T05:47:56.695 INFO:teuthology.orchestra.run.smithi040.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-22T05:47:56.700 INFO:teuthology.orchestra.run.smithi040.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-22T05:47:56.705 INFO:teuthology.orchestra.run.smithi062.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-22T05:47:56.706 INFO:teuthology.orchestra.run.smithi062.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-22T05:47:56.707 INFO:teuthology.orchestra.run.smithi062.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-22T05:47:56.720 INFO:teuthology.orchestra.run.smithi040.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-22T05:47:56.721 INFO:teuthology.orchestra.run.smithi040.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-22T05:47:56.723 INFO:teuthology.orchestra.run.smithi040.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-22T05:47:56.731 INFO:teuthology.orchestra.run.smithi040.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-22T05:47:56.731 INFO:teuthology.orchestra.run.smithi040.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-22T05:47:56.732 INFO:teuthology.orchestra.run.smithi040.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-22T05:47:57.202 INFO:teuthology.orchestra.run.smithi062.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.2-1237-g646b8c7d-1jammy [3,106 kB] 2024-04-22T05:47:57.599 INFO:teuthology.orchestra.run.smithi040.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.2-1237-g646b8c7d-1jammy [5,002 kB] 2024-04-22T05:47:57.684 INFO:teuthology.orchestra.run.smithi040.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.2-1237-g646b8c7d-1jammy [241 kB] 2024-04-22T05:47:57.691 INFO:teuthology.orchestra.run.smithi040.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1237-g646b8c7d-1jammy [130 kB] 2024-04-22T05:47:57.699 INFO:teuthology.orchestra.run.smithi040.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.2-1237-g646b8c7d-1jammy [1,490 kB] 2024-04-22T05:47:57.753 INFO:teuthology.orchestra.run.smithi040.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.2-1237-g646b8c7d-1jammy [6,079 kB] 2024-04-22T05:47:57.982 INFO:teuthology.orchestra.run.smithi040.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.2-1237-g646b8c7d-1jammy [22.3 MB] 2024-04-22T05:47:58.017 INFO:teuthology.orchestra.run.smithi062.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.2-1237-g646b8c7d-1jammy [680 kB] 2024-04-22T05:47:58.203 INFO:teuthology.orchestra.run.smithi062.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.2-1237-g646b8c7d-1jammy [356 kB] 2024-04-22T05:47:58.299 INFO:teuthology.orchestra.run.smithi062.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.2-1237-g646b8c7d-1jammy [32.8 kB] 2024-04-22T05:47:58.316 INFO:teuthology.orchestra.run.smithi062.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.2-1237-g646b8c7d-1jammy [180 kB] 2024-04-22T05:47:58.352 INFO:teuthology.orchestra.run.smithi062.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.2-1237-g646b8c7d-1jammy [63.7 kB] 2024-04-22T05:47:58.371 INFO:teuthology.orchestra.run.smithi062.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.2-1237-g646b8c7d-1jammy [330 kB] 2024-04-22T05:47:58.463 INFO:teuthology.orchestra.run.smithi062.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.2-1237-g646b8c7d-1jammy [5,891 kB] 2024-04-22T05:47:58.919 INFO:teuthology.orchestra.run.smithi040.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.2-1237-g646b8c7d-1jammy [14.1 kB] 2024-04-22T05:47:58.920 INFO:teuthology.orchestra.run.smithi040.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.2-1237-g646b8c7d-1jammy [831 kB] 2024-04-22T05:47:58.936 INFO:teuthology.orchestra.run.smithi040.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.2-1237-g646b8c7d-1jammy [2,154 kB] 2024-04-22T05:47:58.976 INFO:teuthology.orchestra.run.smithi040.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.2-1237-g646b8c7d-1jammy [238 kB] 2024-04-22T05:47:58.980 INFO:teuthology.orchestra.run.smithi040.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.2-1237-g646b8c7d-1jammy [144 kB] 2024-04-22T05:47:58.982 INFO:teuthology.orchestra.run.smithi040.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.2-1237-g646b8c7d-1jammy [2,238 kB] 2024-04-22T05:47:59.026 INFO:teuthology.orchestra.run.smithi040.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1237-g646b8c7d-1jammy [8,625 kB] 2024-04-22T05:47:59.352 INFO:teuthology.orchestra.run.smithi040.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.2-1237-g646b8c7d-1jammy [14.3 kB] 2024-04-22T05:47:59.353 INFO:teuthology.orchestra.run.smithi040.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.2-1237-g646b8c7d-1jammy [42.6 MB] 2024-04-22T05:47:59.982 INFO:teuthology.orchestra.run.smithi062.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.2-1237-g646b8c7d-1jammy [112 kB] 2024-04-22T05:48:00.008 INFO:teuthology.orchestra.run.smithi062.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.2-1237-g646b8c7d-1jammy [441 kB] 2024-04-22T05:48:00.131 INFO:teuthology.orchestra.run.smithi062.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.2-1237-g646b8c7d-1jammy [21.9 MB] 2024-04-22T05:48:01.109 INFO:teuthology.orchestra.run.smithi040.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.2-1237-g646b8c7d-1jammy [121 kB] 2024-04-22T05:48:01.111 INFO:teuthology.orchestra.run.smithi040.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.2-1237-g646b8c7d-1jammy [38.3 kB] 2024-04-22T05:48:01.180 INFO:teuthology.orchestra.run.smithi040.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.2-1237-g646b8c7d-1jammy [11.6 MB] 2024-04-22T05:48:01.565 INFO:teuthology.orchestra.run.smithi040.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.2-1237-g646b8c7d-1jammy [93.0 kB] 2024-04-22T05:48:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 162 MB in 6s (28.8 MB/s) 2024-04-22T05:48:02.268 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libfmt8:amd64. 2024-04-22T05:48:02.361 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-22T05:48:02.366 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-04-22T05:48:02.450 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-04-22T05:48:02.786 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-22T05:48:02.813 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-22T05:48:02.853 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-22T05:48:03.147 INFO:teuthology.orchestra.run.smithi062.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.2-1237-g646b8c7d-1jammy [5,002 kB] 2024-04-22T05:48:03.215 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-22T05:48:03.241 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-22T05:48:03.281 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-22T05:48:03.576 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-22T05:48:03.604 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-22T05:48:03.643 INFO:teuthology.orchestra.run.smithi062.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.2-1237-g646b8c7d-1jammy [241 kB] 2024-04-22T05:48:03.648 INFO:teuthology.orchestra.run.smithi062.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1237-g646b8c7d-1jammy [130 kB] 2024-04-22T05:48:03.679 INFO:teuthology.orchestra.run.smithi062.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.2-1237-g646b8c7d-1jammy [1,490 kB] 2024-04-22T05:48:03.751 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-22T05:48:03.824 INFO:teuthology.orchestra.run.smithi062.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.2-1237-g646b8c7d-1jammy [6,079 kB] 2024-04-22T05:48:04.179 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-22T05:48:04.207 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-22T05:48:04.255 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-22T05:48:04.434 INFO:teuthology.orchestra.run.smithi062.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.2-1237-g646b8c7d-1jammy [22.3 MB] 2024-04-22T05:48:04.649 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-22T05:48:04.677 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-22T05:48:04.716 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-22T05:48:05.036 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-22T05:48:05.063 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-22T05:48:05.094 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-22T05:48:05.472 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-22T05:48:05.499 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-22T05:48:05.539 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-22T05:48:05.867 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librados2. 2024-04-22T05:48:05.893 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../008-librados2_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:05.933 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librados2 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:06.369 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librbd1. 2024-04-22T05:48:06.385 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../009-librbd1_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:06.419 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librbd1 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:06.671 INFO:teuthology.orchestra.run.smithi062.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.2-1237-g646b8c7d-1jammy [14.1 kB] 2024-04-22T05:48:06.672 INFO:teuthology.orchestra.run.smithi062.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.2-1237-g646b8c7d-1jammy [831 kB] 2024-04-22T05:48:06.723 INFO:teuthology.orchestra.run.smithi062.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.2-1237-g646b8c7d-1jammy [2,154 kB] 2024-04-22T05:48:06.814 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libcephfs2. 2024-04-22T05:48:06.841 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../010-libcephfs2_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:06.880 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libcephfs2 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:06.906 INFO:teuthology.orchestra.run.smithi062.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.2-1237-g646b8c7d-1jammy [238 kB] 2024-04-22T05:48:06.938 INFO:teuthology.orchestra.run.smithi062.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.2-1237-g646b8c7d-1jammy [144 kB] 2024-04-22T05:48:06.941 INFO:teuthology.orchestra.run.smithi062.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.2-1237-g646b8c7d-1jammy [2,238 kB] 2024-04-22T05:48:07.153 INFO:teuthology.orchestra.run.smithi062.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1237-g646b8c7d-1jammy [8,625 kB] 2024-04-22T05:48:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rados. 2024-04-22T05:48:07.182 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../011-python3-rados_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:07.224 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rados (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:07.552 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-22T05:48:07.580 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:07.619 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-ceph-argparse (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:07.863 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cephfs. 2024-04-22T05:48:07.887 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../013-python3-cephfs_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:07.907 INFO:teuthology.orchestra.run.smithi062.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.2-1237-g646b8c7d-1jammy [14.3 kB] 2024-04-22T05:48:07.921 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cephfs (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:07.952 INFO:teuthology.orchestra.run.smithi062.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.2-1237-g646b8c7d-1jammy [42.6 MB] 2024-04-22T05:48:08.216 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-22T05:48:08.243 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:08.282 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-ceph-common (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:08.611 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-22T05:48:08.638 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-22T05:48:08.677 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-22T05:48:09.079 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-prettytable. 2024-04-22T05:48:09.106 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-04-22T05:48:09.155 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-22T05:48:09.416 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rbd. 2024-04-22T05:48:09.432 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../017-python3-rbd_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:09.466 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rbd (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:09.811 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-22T05:48:09.838 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-22T05:48:09.877 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-22T05:48:10.197 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librgw2. 2024-04-22T05:48:10.224 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../019-librgw2_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:10.264 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librgw2 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:10.749 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rgw. 2024-04-22T05:48:10.765 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../020-python3-rgw_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:10.799 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rgw (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:11.127 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-22T05:48:11.154 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-22T05:48:11.183 INFO:teuthology.orchestra.run.smithi062.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.2-1237-g646b8c7d-1jammy [121 kB] 2024-04-22T05:48:11.186 INFO:teuthology.orchestra.run.smithi062.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.2-1237-g646b8c7d-1jammy [38.3 kB] 2024-04-22T05:48:11.187 INFO:teuthology.orchestra.run.smithi062.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.2-1237-g646b8c7d-1jammy [11.6 MB] 2024-04-22T05:48:11.194 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-22T05:48:11.497 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libradosstriper1. 2024-04-22T05:48:11.524 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../022-libradosstriper1_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:11.563 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libradosstriper1 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:11.684 INFO:teuthology.orchestra.run.smithi062.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.2-1237-g646b8c7d-1jammy [93.0 kB] 2024-04-22T05:48:11.950 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-common. 2024-04-22T05:48:11.977 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../023-ceph-common_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:12.016 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-common (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:12.273 INFO:teuthology.orchestra.run.smithi062.stdout:Fetched 162 MB in 16s (10.3 MB/s) 2024-04-22T05:48:12.531 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libfmt8:amd64. 2024-04-22T05:48:12.635 INFO:teuthology.orchestra.run.smithi062.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-22T05:48:12.640 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-04-22T05:48:12.738 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-04-22T05:48:12.999 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-22T05:48:13.026 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-22T05:48:13.057 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-22T05:48:13.298 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-base. 2024-04-22T05:48:13.324 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../024-ceph-base_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:13.378 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-22T05:48:13.404 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-22T05:48:13.444 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-22T05:48:13.627 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-base (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:14.066 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-22T05:48:14.093 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-22T05:48:14.219 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-22T05:48:14.246 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-22T05:48:14.247 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-22T05:48:14.285 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-22T05:48:14.597 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cheroot. 2024-04-22T05:48:14.625 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-22T05:48:14.634 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-22T05:48:14.662 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-22T05:48:14.671 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-22T05:48:14.701 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-22T05:48:14.983 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-22T05:48:15.010 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-22T05:48:15.041 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-22T05:48:15.104 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-22T05:48:15.131 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-22T05:48:15.171 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-22T05:48:15.311 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-22T05:48:15.338 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-22T05:48:15.377 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-22T05:48:15.541 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-22T05:48:15.568 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-22T05:48:15.607 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-22T05:48:15.689 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-22T05:48:15.716 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-22T05:48:15.755 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-22T05:48:15.919 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-22T05:48:15.945 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-22T05:48:15.985 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-22T05:48:16.100 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-tempora. 2024-04-22T05:48:16.127 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-04-22T05:48:16.183 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-22T05:48:16.297 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package librados2. 2024-04-22T05:48:16.323 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../008-librados2_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:16.363 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking librados2 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:16.503 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-portend. 2024-04-22T05:48:16.529 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-04-22T05:48:16.569 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-22T05:48:16.774 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package librbd1. 2024-04-22T05:48:16.801 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../009-librbd1_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:16.833 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking librbd1 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:16.873 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-22T05:48:16.899 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-22T05:48:16.939 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-22T05:48:17.236 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libcephfs2. 2024-04-22T05:48:17.262 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../010-libcephfs2_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:17.292 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-22T05:48:17.302 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libcephfs2 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:17.319 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-22T05:48:17.367 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-22T05:48:17.588 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-rados. 2024-04-22T05:48:17.603 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../011-python3-rados_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:17.646 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-rados (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:17.720 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-natsort. 2024-04-22T05:48:17.747 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-04-22T05:48:17.787 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-22T05:48:17.966 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-22T05:48:17.992 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:18.032 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-ceph-argparse (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:18.115 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-logutils. 2024-04-22T05:48:18.142 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-04-22T05:48:18.182 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-22T05:48:18.260 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cephfs. 2024-04-22T05:48:18.275 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../013-python3-cephfs_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:18.335 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cephfs (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:18.552 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-22T05:48:18.578 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-22T05:48:18.635 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-22T05:48:18.637 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-22T05:48:18.664 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:18.704 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-ceph-common (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:18.954 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-mako. 2024-04-22T05:48:18.981 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-22T05:48:19.021 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-22T05:48:19.024 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-22T05:48:19.051 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-22T05:48:19.208 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-22T05:48:19.316 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-22T05:48:19.332 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-22T05:48:19.366 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-22T05:48:19.519 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-prettytable. 2024-04-22T05:48:19.546 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-04-22T05:48:19.585 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-22T05:48:19.652 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-22T05:48:19.679 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-22T05:48:19.719 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-22T05:48:19.821 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-rbd. 2024-04-22T05:48:19.837 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../017-python3-rbd_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:19.871 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-rbd (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:20.021 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-webob. 2024-04-22T05:48:20.048 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-22T05:48:20.088 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-22T05:48:20.201 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-22T05:48:20.228 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-22T05:48:20.268 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-22T05:48:20.450 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-waitress. 2024-04-22T05:48:20.476 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-22T05:48:20.519 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-22T05:48:20.587 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package librgw2. 2024-04-22T05:48:20.614 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../019-librgw2_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:20.653 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking librgw2 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:20.811 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-tempita. 2024-04-22T05:48:20.826 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-22T05:48:20.861 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-22T05:48:21.130 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-paste. 2024-04-22T05:48:21.157 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-22T05:48:21.181 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-rgw. 2024-04-22T05:48:21.196 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../020-python3-rgw_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:21.197 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-22T05:48:21.230 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-rgw (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:21.517 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-22T05:48:21.544 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-22T05:48:21.575 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-22T05:48:21.575 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-22T05:48:21.602 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-22T05:48:21.642 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-22T05:48:21.836 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-22T05:48:21.863 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-22T05:48:21.903 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-22T05:48:21.953 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libradosstriper1. 2024-04-22T05:48:21.980 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../022-libradosstriper1_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:22.019 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libradosstriper1 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:22.231 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-webtest. 2024-04-22T05:48:22.258 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-04-22T05:48:22.297 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-22T05:48:22.439 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-common. 2024-04-22T05:48:22.466 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../023-ceph-common_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:22.506 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-common (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:22.609 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pecan. 2024-04-22T05:48:22.636 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-22T05:48:22.675 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-22T05:48:23.047 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-certifi. 2024-04-22T05:48:23.074 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-04-22T05:48:23.113 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-22T05:48:23.408 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-idna. 2024-04-22T05:48:23.424 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../049-python3-idna_3.3-1_all.deb ... 2024-04-22T05:48:23.458 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-22T05:48:23.794 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-urllib3. 2024-04-22T05:48:23.803 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-base. 2024-04-22T05:48:23.821 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-22T05:48:23.830 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../024-ceph-base_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:23.879 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-base (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:24.053 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-22T05:48:24.381 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-requests. 2024-04-22T05:48:24.399 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-22T05:48:24.408 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-22T05:48:24.426 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-22T05:48:24.448 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-22T05:48:24.482 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-22T05:48:24.759 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-22T05:48:24.785 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cheroot. 2024-04-22T05:48:24.786 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-22T05:48:24.812 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-22T05:48:24.825 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-22T05:48:24.851 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-22T05:48:25.120 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-22T05:48:25.147 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:25.171 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-22T05:48:25.187 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:25.198 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-22T05:48:25.237 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-22T05:48:25.549 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-22T05:48:25.550 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-22T05:48:25.575 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-22T05:48:25.577 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:25.608 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:25.615 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-22T05:48:25.885 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-22T05:48:25.886 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-openssl. 2024-04-22T05:48:25.900 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-22T05:48:25.913 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../055-python3-openssl_21.0.0-1_all.deb ... 2024-04-22T05:48:25.934 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-22T05:48:25.952 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-22T05:48:26.237 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-tempora. 2024-04-22T05:48:26.264 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-04-22T05:48:26.297 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr. 2024-04-22T05:48:26.304 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-22T05:48:26.324 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../056-ceph-mgr_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:26.364 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:26.623 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-portend. 2024-04-22T05:48:26.650 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-04-22T05:48:26.690 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-22T05:48:26.726 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mon. 2024-04-22T05:48:26.742 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../057-ceph-mon_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:26.776 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mon (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:26.976 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-22T05:48:27.003 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-22T05:48:27.042 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-22T05:48:27.304 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-22T05:48:27.330 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-22T05:48:27.363 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-osd. 2024-04-22T05:48:27.370 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-22T05:48:27.389 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../058-ceph-osd_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:27.429 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-osd (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:27.757 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-natsort. 2024-04-22T05:48:27.784 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-04-22T05:48:27.823 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-22T05:48:28.110 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-logutils. 2024-04-22T05:48:28.126 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-04-22T05:48:28.160 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-22T05:48:28.541 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph. 2024-04-22T05:48:28.568 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../059-ceph_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:28.607 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:28.927 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-fuse. 2024-04-22T05:48:28.954 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../060-ceph-fuse_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:28.993 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-fuse (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:29.105 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-22T05:48:29.132 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-22T05:48:29.171 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-22T05:48:29.674 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-mako. 2024-04-22T05:48:29.701 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-22T05:48:29.749 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-22T05:48:30.077 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-22T05:48:30.104 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-22T05:48:30.144 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-22T05:48:30.164 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mds. 2024-04-22T05:48:30.192 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../061-ceph-mds_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:30.230 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mds (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:30.430 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-22T05:48:30.456 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-22T05:48:30.488 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-22T05:48:30.626 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephadm. 2024-04-22T05:48:30.652 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../062-cephadm_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:30.692 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephadm (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:30.782 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-webob. 2024-04-22T05:48:30.809 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-22T05:48:30.849 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-22T05:48:30.979 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-22T05:48:31.005 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../063-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-22T05:48:31.045 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-22T05:48:31.194 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-waitress. 2024-04-22T05:48:31.220 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-22T05:48:31.263 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-22T05:48:31.356 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-22T05:48:31.386 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../064-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-22T05:48:31.422 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-22T05:48:31.571 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-tempita. 2024-04-22T05:48:31.598 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-22T05:48:31.638 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-22T05:48:31.940 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-paste. 2024-04-22T05:48:31.967 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-22T05:48:32.007 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-22T05:48:32.327 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-22T05:48:32.354 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-22T05:48:32.393 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-22T05:48:32.420 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-babel. 2024-04-22T05:48:32.447 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../065-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-22T05:48:32.486 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-22T05:48:32.696 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-22T05:48:32.723 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-22T05:48:32.762 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-22T05:48:32.839 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jinja2. 2024-04-22T05:48:32.867 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../066-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-22T05:48:32.906 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-22T05:48:33.040 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-webtest. 2024-04-22T05:48:33.068 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-04-22T05:48:33.107 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-22T05:48:33.261 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-22T05:48:33.286 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../067-ceph-mgr-cephadm_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:33.328 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:33.402 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pecan. 2024-04-22T05:48:33.431 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-22T05:48:33.459 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-22T05:48:33.665 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-22T05:48:33.692 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../068-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-22T05:48:33.731 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-22T05:48:33.789 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-certifi. 2024-04-22T05:48:33.805 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-04-22T05:48:33.848 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-22T05:48:34.026 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pluggy. 2024-04-22T05:48:34.053 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-22T05:48:34.167 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-idna. 2024-04-22T05:48:34.194 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../049-python3-idna_3.3-1_all.deb ... 2024-04-22T05:48:34.233 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-22T05:48:34.238 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-22T05:48:34.520 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-urllib3. 2024-04-22T05:48:34.546 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-22T05:48:34.586 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-22T05:48:34.612 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-py. 2024-04-22T05:48:34.640 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-04-22T05:48:34.678 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-22T05:48:34.906 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-requests. 2024-04-22T05:48:34.933 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-22T05:48:34.972 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-22T05:48:35.007 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-22T05:48:35.034 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-22T05:48:35.073 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-22T05:48:35.284 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-22T05:48:35.310 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-22T05:48:35.350 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-22T05:48:35.410 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-toml. 2024-04-22T05:48:35.437 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-04-22T05:48:35.476 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-22T05:48:35.636 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-22T05:48:35.652 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:35.694 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:35.729 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pytest. 2024-04-22T05:48:35.746 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-22T05:48:35.779 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-22T05:48:36.082 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-22T05:48:36.108 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-22T05:48:36.109 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:36.136 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-04-22T05:48:36.149 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:36.174 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-22T05:48:36.435 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-openssl. 2024-04-22T05:48:36.462 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../055-python3-openssl_21.0.0-1_all.deb ... 2024-04-22T05:48:36.486 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-routes. 2024-04-22T05:48:36.501 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-22T05:48:36.513 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-22T05:48:36.552 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-22T05:48:36.846 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr. 2024-04-22T05:48:36.873 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../056-ceph-mgr_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:36.889 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-22T05:48:36.912 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:36.917 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:36.955 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:37.266 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mon. 2024-04-22T05:48:37.292 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../057-ceph-mon_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:37.324 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mon (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:37.927 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-osd. 2024-04-22T05:48:37.954 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../058-ceph-osd_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:37.994 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-osd (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:38.177 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-22T05:48:38.193 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-22T05:48:38.227 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-22T05:48:38.739 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-joblib. 2024-04-22T05:48:38.767 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-22T05:48:38.805 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-22T05:48:39.013 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph. 2024-04-22T05:48:39.029 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../059-ceph_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:39.071 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:39.193 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-22T05:48:39.220 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-22T05:48:39.259 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-22T05:48:39.374 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-fuse. 2024-04-22T05:48:39.401 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../060-ceph-fuse_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:39.432 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-fuse (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:39.537 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-sklearn. 2024-04-22T05:48:39.565 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-22T05:48:39.603 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-22T05:48:39.853 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mds. 2024-04-22T05:48:39.879 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../061-ceph-mds_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:39.919 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mds (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:40.304 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-22T05:48:40.305 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package cephadm. 2024-04-22T05:48:40.332 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../062-cephadm_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:40.333 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:40.370 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:40.371 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking cephadm (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:40.683 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-22T05:48:40.710 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../063-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-22T05:48:40.749 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-22T05:48:41.061 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-22T05:48:41.077 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../064-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-22T05:48:41.118 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-22T05:48:41.315 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cachetools. 2024-04-22T05:48:41.343 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-04-22T05:48:41.381 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-22T05:48:41.744 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-22T05:48:41.772 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-22T05:48:41.802 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-22T05:48:42.090 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-babel. 2024-04-22T05:48:42.097 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-22T05:48:42.106 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../065-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-22T05:48:42.114 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-22T05:48:42.140 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-22T05:48:42.155 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-22T05:48:42.434 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rsa. 2024-04-22T05:48:42.443 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-jinja2. 2024-04-22T05:48:42.461 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-04-22T05:48:42.470 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../066-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-22T05:48:42.500 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-22T05:48:42.509 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-22T05:48:42.821 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-google-auth. 2024-04-22T05:48:42.848 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-04-22T05:48:42.887 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-22T05:48:42.905 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-22T05:48:42.932 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../067-ceph-mgr-cephadm_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:42.971 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:43.232 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-22T05:48:43.260 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-22T05:48:43.291 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-22T05:48:43.298 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-22T05:48:43.318 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../068-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-22T05:48:43.357 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-22T05:48:43.618 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-websocket. 2024-04-22T05:48:43.646 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-04-22T05:48:43.669 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pluggy. 2024-04-22T05:48:43.685 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-22T05:48:43.696 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-22T05:48:43.735 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-22T05:48:43.946 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-22T05:48:43.962 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-22T05:48:43.996 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-22T05:48:44.046 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-py. 2024-04-22T05:48:44.074 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-04-22T05:48:44.113 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-22T05:48:44.466 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-22T05:48:44.494 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-22T05:48:44.533 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-22T05:48:44.619 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-22T05:48:44.636 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:44.678 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:44.794 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-toml. 2024-04-22T05:48:44.810 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-04-22T05:48:44.844 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-22T05:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-22T05:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-22T05:48:45.063 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-22T05:48:45.355 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pytest. 2024-04-22T05:48:45.382 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-22T05:48:45.399 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-22T05:48:45.421 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-22T05:48:45.427 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-22T05:48:45.465 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-22T05:48:45.702 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package jq. 2024-04-22T05:48:45.718 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-22T05:48:45.751 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-22T05:48:45.758 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-22T05:48:45.774 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-04-22T05:48:45.808 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-22T05:48:45.979 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package socat. 2024-04-22T05:48:46.004 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-22T05:48:46.046 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-22T05:48:46.136 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-routes. 2024-04-22T05:48:46.164 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-22T05:48:46.203 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-22T05:48:46.365 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package xmlstarlet. 2024-04-22T05:48:46.382 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-22T05:48:46.424 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-22T05:48:46.514 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-22T05:48:46.542 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:46.572 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:46.803 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-test. 2024-04-22T05:48:46.820 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../096-ceph-test_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:46.861 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-test (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:47.770 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-22T05:48:47.786 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-22T05:48:47.827 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-22T05:48:48.323 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-joblib. 2024-04-22T05:48:48.350 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-22T05:48:48.389 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-22T05:48:48.743 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-22T05:48:48.765 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-volume. 2024-04-22T05:48:48.770 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-22T05:48:48.793 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../097-ceph-volume_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:48.809 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-22T05:48:48.823 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-volume (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:49.077 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-22T05:48:49.094 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:49.112 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-sklearn. 2024-04-22T05:48:49.127 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libcephfs-dev (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:49.139 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-22T05:48:49.178 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-22T05:48:49.497 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package nvme-cli. 2024-04-22T05:48:49.525 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-22T05:48:49.564 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-22T05:48:50.037 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-22T05:48:50.065 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:50.103 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:50.494 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-22T05:48:50.522 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-22T05:48:50.560 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-22T05:48:51.005 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pastescript. 2024-04-22T05:48:51.033 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-04-22T05:48:51.071 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-22T05:48:51.148 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cachetools. 2024-04-22T05:48:51.175 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-04-22T05:48:51.214 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-22T05:48:51.416 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-psutil. 2024-04-22T05:48:51.445 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-22T05:48:51.483 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-22T05:48:51.602 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-22T05:48:51.630 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-22T05:48:51.668 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-22T05:48:51.836 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pygments. 2024-04-22T05:48:51.866 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-22T05:48:51.902 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-22T05:48:51.997 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-22T05:48:52.024 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-22T05:48:52.063 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-22T05:48:52.381 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-22T05:48:52.383 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-rsa. 2024-04-22T05:48:52.410 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-22T05:48:52.410 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-04-22T05:48:52.448 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-22T05:48:52.449 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-22T05:48:52.709 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-simplejson. 2024-04-22T05:48:52.726 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-22T05:48:52.728 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-google-auth. 2024-04-22T05:48:52.745 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-04-22T05:48:52.758 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-22T05:48:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-22T05:48:53.020 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-22T05:48:53.037 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-22T05:48:53.048 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-22T05:48:53.075 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-22T05:48:53.078 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-22T05:48:53.114 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-22T05:48:53.425 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-websocket. 2024-04-22T05:48:53.453 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-04-22T05:48:53.492 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-22T05:48:53.749 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package radosgw. 2024-04-22T05:48:53.777 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../107-radosgw_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:53.804 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-22T05:48:53.815 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking radosgw (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:53.831 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-22T05:48:53.870 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-22T05:48:54.559 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package rbd-fuse. 2024-04-22T05:48:54.567 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-22T05:48:54.585 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../108-rbd-fuse_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:54.585 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:54.625 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:54.626 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking rbd-fuse (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:54.939 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-22T05:48:54.944 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-22T05:48:54.972 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-22T05:48:55.002 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-22T05:48:55.214 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-22T05:48:55.356 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-22T05:48:55.384 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-22T05:48:55.422 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-22T05:48:55.497 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-22T05:48:55.674 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package jq. 2024-04-22T05:48:55.691 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-22T05:48:55.732 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-22T05:48:55.815 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-22T05:48:55.994 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package socat. 2024-04-22T05:48:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-22T05:48:56.052 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-22T05:48:56.161 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-22T05:48:56.347 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package xmlstarlet. 2024-04-22T05:48:56.374 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-22T05:48:56.413 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-22T05:48:56.603 INFO:teuthology.orchestra.run.smithi040.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-22T05:48:56.684 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephadm (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:56.770 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-test. 2024-04-22T05:48:56.784 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../096-ceph-test_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:56.825 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-test (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:57.318 INFO:teuthology.orchestra.run.smithi040.stdout:Adding system user cephadm....done 2024-04-22T05:48:57.458 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-22T05:48:58.118 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-22T05:48:58.395 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-22T05:48:58.521 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-22T05:48:58.763 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-volume. 2024-04-22T05:48:58.771 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-22T05:48:58.791 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../097-ceph-volume_18.2.2-1237-g646b8c7d-1jammy_all.deb ... 2024-04-22T05:48:58.829 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-volume (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:59.023 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-22T05:48:59.133 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-22T05:48:59.149 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-22T05:48:59.150 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:48:59.191 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libcephfs-dev (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:48:59.442 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-22T05:48:59.572 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package nvme-cli. 2024-04-22T05:48:59.599 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-22T05:48:59.629 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-22T05:48:59.819 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-22T05:48:59.934 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-22T05:48:59.962 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-22T05:49:00.000 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-22T05:49:00.078 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-22T05:49:00.338 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-22T05:49:00.412 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pastescript. 2024-04-22T05:49:00.440 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-04-22T05:49:00.478 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-22T05:49:00.581 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-22T05:49:00.815 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-psutil. 2024-04-22T05:49:00.843 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-22T05:49:00.848 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-22T05:49:00.881 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-22T05:49:01.125 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-ceph-argparse (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:01.367 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-22T05:49:01.451 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pygments. 2024-04-22T05:49:01.479 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-22T05:49:01.493 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-22T05:49:01.517 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-22T05:49:01.753 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-22T05:49:01.879 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-22T05:49:01.972 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-22T05:49:02.000 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-22T05:49:02.038 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-22T05:49:02.247 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-22T05:49:02.348 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-22T05:49:02.349 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-simplejson. 2024-04-22T05:49:02.378 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-22T05:49:02.415 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-22T05:49:02.649 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-22T05:49:02.685 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-22T05:49:02.701 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-22T05:49:02.743 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-22T05:49:02.917 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-22T05:49:03.234 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-22T05:49:03.514 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package radosgw. 2024-04-22T05:49:03.542 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../107-radosgw_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:49:03.580 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking radosgw (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:03.911 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-22T05:49:04.179 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-22T05:49:04.280 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-22T05:49:04.291 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package rbd-fuse. 2024-04-22T05:49:04.320 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../108-rbd-fuse_18.2.2-1237-g646b8c7d-1jammy_amd64.deb ... 2024-04-22T05:49:04.378 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking rbd-fuse (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:04.381 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-22T05:49:04.657 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-22T05:49:04.682 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-22T05:49:04.937 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-22T05:49:05.063 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-22T05:49:05.100 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-22T05:49:05.344 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-22T05:49:05.351 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-22T05:49:05.666 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-22T05:49:05.770 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-22T05:49:06.068 INFO:teuthology.orchestra.run.smithi062.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-22T05:49:06.070 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-22T05:49:06.148 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up cephadm (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:06.431 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-22T05:49:06.681 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-22T05:49:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:Adding system user cephadm....done 2024-04-22T05:49:06.799 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-ceph-common (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-22T05:49:07.092 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-idna (3.3-1) ... 2024-04-22T05:49:07.113 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-22T05:49:07.356 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-22T05:49:07.418 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-22T05:49:07.516 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-22T05:49:07.775 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-22T05:49:08.043 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-22T05:49:08.169 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-22T05:49:08.495 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-22T05:49:08.615 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-22T05:49:08.750 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-22T05:49:08.864 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-22T05:49:09.001 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-22T05:49:09.115 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-22T05:49:09.127 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-22T05:49:09.254 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-22T05:49:09.391 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-22T05:49:09.604 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-22T05:49:09.642 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-22T05:49:09.881 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-22T05:49:09.901 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-22T05:49:10.123 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-22T05:49:10.160 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-ceph-argparse (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:10.249 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-04-22T05:49:10.375 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-22T05:49:10.437 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-22T05:49:10.588 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-22T05:49:10.685 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-22T05:49:10.847 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-22T05:49:10.948 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-22T05:49:10.961 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-22T05:49:11.212 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-22T05:49:11.316 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-22T05:49:11.442 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-22T05:49:11.455 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-22T05:49:11.706 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-22T05:49:11.718 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-22T05:49:11.977 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-22T05:49:11.982 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-22T05:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-22T05:49:12.303 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-22T05:49:12.350 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-22T05:49:12.485 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-22T05:49:12.778 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-22T05:49:13.022 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-22T05:49:13.087 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-22T05:49:13.281 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-22T05:49:13.339 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-22T05:49:13.407 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-22T05:49:13.533 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-22T05:49:13.721 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-22T05:49:13.868 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-22T05:49:14.008 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-22T05:49:14.109 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-22T05:49:14.269 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-22T05:49:14.502 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-22T05:49:14.528 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-22T05:49:14.762 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-22T05:49:14.913 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-22T05:49:15.005 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-22T05:49:15.106 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-22T05:49:15.206 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-22T05:49:15.423 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-22T05:49:15.532 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-22T05:49:15.733 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-22T05:49:15.791 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-22T05:49:15.917 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-ceph-common (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:16.001 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-22T05:49:16.160 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-22T05:49:16.218 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-idna (3.3-1) ... 2024-04-22T05:49:16.436 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-22T05:49:16.562 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-22T05:49:16.712 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-22T05:49:16.997 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-22T05:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-22T05:49:17.666 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-22T05:49:17.825 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-22T05:49:17.951 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-22T05:49:18.026 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-22T05:49:18.152 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librados2 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:18.219 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-22T05:49:18.279 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librgw2 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:18.354 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-22T05:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:18.480 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-22T05:49:18.489 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libcephfs2 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:18.590 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libradosstriper1 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:18.781 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-22T05:49:18.801 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-22T05:49:19.023 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-22T05:49:19.103 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librbd1 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:19.229 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-modules-core (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:19.291 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-22T05:49:19.355 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-fuse (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:19.417 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-04-22T05:49:19.543 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-22T05:49:19.577 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-22T05:49:19.577 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-22T05:49:19.861 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-22T05:49:20.098 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libcephfs-dev (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:20.162 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-22T05:49:20.224 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rados (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:20.350 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-22T05:49:20.430 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-22T05:49:20.681 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-22T05:49:20.915 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-22T05:49:21.191 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-22T05:49:21.317 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-22T05:49:21.557 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rbd (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:21.618 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-22T05:49:21.682 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up rbd-fuse (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:21.744 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-22T05:49:21.809 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rgw (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:21.935 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cephfs (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:22.061 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-common (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:22.146 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-22T05:49:22.446 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-22T05:49:22.595 INFO:teuthology.orchestra.run.smithi040.stdout:Adding group ceph....done 2024-04-22T05:49:22.706 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-22T05:49:22.972 INFO:teuthology.orchestra.run.smithi040.stdout:Adding system user ceph....done 2024-04-22T05:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout:Setting system user ceph properties....done 2024-04-22T05:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-22T05:49:23.202 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-22T05:49:23.321 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-22T05:49:23.508 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-22T05:49:23.626 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-22T05:49:23.686 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-22T05:49:24.011 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-22T05:49:24.204 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-test (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:24.287 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-22T05:49:24.330 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up radosgw (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:24.546 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-22T05:49:24.665 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-22T05:49:24.842 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-22T05:49:24.842 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-22T05:49:24.965 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-22T05:49:25.283 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-22T05:49:25.323 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-base (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:25.567 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-22T05:49:25.672 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-22T05:49:25.693 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-22T05:49:25.969 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-22T05:49:26.158 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mds (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:26.254 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-22T05:49:26.425 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-22T05:49:26.425 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-22T05:49:26.530 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-22T05:49:26.894 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:26.914 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-22T05:49:27.125 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-22T05:49:27.125 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-22T05:49:27.191 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-22T05:49:27.534 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-22T05:49:27.635 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up librados2 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:27.670 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-osd (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:27.736 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up librgw2 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:27.862 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:27.944 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-22T05:49:27.944 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-22T05:49:27.988 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libcephfs2 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:28.114 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libradosstriper1 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:28.325 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-22T05:49:28.422 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:28.523 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:28.611 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up librbd1 (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:28.657 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mon (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:28.736 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-modules-core (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:28.871 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-fuse (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:28.874 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-22T05:49:28.874 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-22T05:49:29.114 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-22T05:49:29.114 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-22T05:49:29.358 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-cephadm (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:29.509 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:29.581 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libcephfs-dev (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:29.635 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-dashboard (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:29.715 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-rados (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:29.786 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-volume (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:29.892 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-22T05:49:30.156 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-22T05:49:31.121 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-rbd (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:31.247 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up rbd-fuse (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:31.373 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-rgw (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:31.407 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-22T05:49:31.499 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cephfs (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:31.625 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-common (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:31.879 INFO:teuthology.orchestra.run.smithi040.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-22T05:49:31.883 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install sqlite3 2024-04-22T05:49:31.944 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-22T05:49:32.126 INFO:teuthology.orchestra.run.smithi062.stdout:Adding group ceph....done 2024-04-22T05:49:32.149 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2024-04-22T05:49:32.150 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2024-04-22T05:49:32.346 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2024-04-22T05:49:32.346 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-22T05:49:32.346 INFO:teuthology.orchestra.run.smithi040.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-22T05:49:32.347 INFO:teuthology.orchestra.run.smithi040.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-22T05:49:32.347 INFO:teuthology.orchestra.run.smithi040.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-22T05:49:32.347 INFO:teuthology.orchestra.run.smithi040.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-22T05:49:32.347 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-22T05:49:32.348 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2024-04-22T05:49:32.349 INFO:teuthology.orchestra.run.smithi040.stdout: libsqlite3-0 2024-04-22T05:49:32.350 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2024-04-22T05:49:32.350 INFO:teuthology.orchestra.run.smithi040.stdout: sqlite3-doc 2024-04-22T05:49:32.382 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2024-04-22T05:49:32.382 INFO:teuthology.orchestra.run.smithi040.stdout: sqlite3 2024-04-22T05:49:32.383 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages will be upgraded: 2024-04-22T05:49:32.384 INFO:teuthology.orchestra.run.smithi040.stdout: libsqlite3-0 2024-04-22T05:49:32.478 INFO:teuthology.orchestra.run.smithi062.stdout:Adding system user ceph....done 2024-04-22T05:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout:1 upgraded, 1 newly installed, 0 to remove and 301 not upgraded. 2024-04-22T05:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 1,410 kB of archives. 2024-04-22T05:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 1,873 kB of additional disk space will be used. 2024-04-22T05:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libsqlite3-0 amd64 3.37.2-2ubuntu0.3 [641 kB] 2024-04-22T05:49:32.638 INFO:teuthology.orchestra.run.smithi062.stdout:Setting system user ceph properties....done 2024-04-22T05:49:32.655 INFO:teuthology.orchestra.run.smithi062.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-22T05:49:32.798 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-22T05:49:33.133 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-22T05:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 sqlite3 amd64 3.37.2-2ubuntu0.3 [768 kB] 2024-04-22T05:49:33.584 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-test (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 1,410 kB in 1s (1,678 kB/s) 2024-04-22T05:49:33.710 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up radosgw (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:33.877 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173143 files and directories currently installed.) 2024-04-22T05:49:33.883 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../libsqlite3-0_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-22T05:49:34.046 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) over (3.37.2-2ubuntu0.1) ... 2024-04-22T05:49:34.179 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-22T05:49:34.179 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-22T05:49:34.366 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package sqlite3. 2024-04-22T05:49:34.395 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../sqlite3_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-22T05:49:34.433 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-22T05:49:34.587 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-base (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:34.809 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) ... 2024-04-22T05:49:34.937 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-22T05:49:34.940 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-22T05:49:35.064 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-22T05:49:35.356 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mds (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:35.601 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-22T05:49:35.602 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-22T05:49:36.041 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:36.072 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-22T05:49:36.310 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-22T05:49:36.310 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-22T05:49:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-22T05:49:36.544 DEBUG:teuthology.parallel:result is None 2024-04-22T05:49:36.775 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-osd (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:37.045 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-22T05:49:37.045 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-22T05:49:37.519 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:37.644 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:37.796 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mon (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:38.013 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-22T05:49:38.013 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-22T05:49:38.447 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-cephadm (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:38.590 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:38.699 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-dashboard (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:38.850 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-volume (18.2.2-1237-g646b8c7d-1jammy) ... 2024-04-22T05:49:39.194 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-22T05:49:40.345 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-22T05:49:40.819 INFO:teuthology.orchestra.run.smithi062.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-22T05:49:40.822 DEBUG:teuthology.orchestra.run.smithi062:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install sqlite3 2024-04-22T05:49:40.879 INFO:teuthology.orchestra.run.smithi062.stdout:Reading package lists... 2024-04-22T05:49:41.083 INFO:teuthology.orchestra.run.smithi062.stdout:Building dependency tree... 2024-04-22T05:49:41.084 INFO:teuthology.orchestra.run.smithi062.stdout:Reading state information... 2024-04-22T05:49:41.278 INFO:teuthology.orchestra.run.smithi062.stdout:The following packages were automatically installed and are no longer required: 2024-04-22T05:49:41.278 INFO:teuthology.orchestra.run.smithi062.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-22T05:49:41.279 INFO:teuthology.orchestra.run.smithi062.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-22T05:49:41.279 INFO:teuthology.orchestra.run.smithi062.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-22T05:49:41.279 INFO:teuthology.orchestra.run.smithi062.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-22T05:49:41.279 INFO:teuthology.orchestra.run.smithi062.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-22T05:49:41.279 INFO:teuthology.orchestra.run.smithi062.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-22T05:49:41.280 INFO:teuthology.orchestra.run.smithi062.stdout:The following additional packages will be installed: 2024-04-22T05:49:41.281 INFO:teuthology.orchestra.run.smithi062.stdout: libsqlite3-0 2024-04-22T05:49:41.282 INFO:teuthology.orchestra.run.smithi062.stdout:Suggested packages: 2024-04-22T05:49:41.282 INFO:teuthology.orchestra.run.smithi062.stdout: sqlite3-doc 2024-04-22T05:49:41.311 INFO:teuthology.orchestra.run.smithi062.stdout:The following NEW packages will be installed: 2024-04-22T05:49:41.311 INFO:teuthology.orchestra.run.smithi062.stdout: sqlite3 2024-04-22T05:49:41.312 INFO:teuthology.orchestra.run.smithi062.stdout:The following packages will be upgraded: 2024-04-22T05:49:41.313 INFO:teuthology.orchestra.run.smithi062.stdout: libsqlite3-0 2024-04-22T05:49:41.493 INFO:teuthology.orchestra.run.smithi062.stdout:1 upgraded, 1 newly installed, 0 to remove and 301 not upgraded. 2024-04-22T05:49:41.493 INFO:teuthology.orchestra.run.smithi062.stdout:Need to get 1,410 kB of archives. 2024-04-22T05:49:41.493 INFO:teuthology.orchestra.run.smithi062.stdout:After this operation, 1,873 kB of additional disk space will be used. 2024-04-22T05:49:41.494 INFO:teuthology.orchestra.run.smithi062.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libsqlite3-0 amd64 3.37.2-2ubuntu0.3 [641 kB] 2024-04-22T05:49:42.066 INFO:teuthology.orchestra.run.smithi062.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 sqlite3 amd64 3.37.2-2ubuntu0.3 [768 kB] 2024-04-22T05:49:42.541 INFO:teuthology.orchestra.run.smithi062.stdout:Fetched 1,410 kB in 1s (1,683 kB/s) 2024-04-22T05:49:42.807 INFO:teuthology.orchestra.run.smithi062.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173143 files and directories currently installed.) 2024-04-22T05:49:42.813 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../libsqlite3-0_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-22T05:49:43.008 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) over (3.37.2-2ubuntu0.1) ... 2024-04-22T05:49:43.328 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package sqlite3. 2024-04-22T05:49:43.357 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../sqlite3_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-22T05:49:43.395 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-22T05:49:43.713 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) ... 2024-04-22T05:49:43.841 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-22T05:49:43.968 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-22T05:49:45.026 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-22T05:49:45.486 INFO:teuthology.orchestra.run.smithi062.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-22T05:49:45.491 DEBUG:teuthology.parallel:result is None 2024-04-22T05:49:45.492 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-22T05:49:45.856 DEBUG:teuthology.orchestra.run.smithi040:> dpkg-query -W -f '${Version}' ceph 2024-04-22T05:49:45.876 INFO:teuthology.orchestra.run.smithi040.stdout:18.2.2-1237-g646b8c7d-1jammy 2024-04-22T05:49:45.877 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1237-g646b8c7d-1jammy 2024-04-22T05:49:45.877 INFO:teuthology.task.install:The correct ceph version 18.2.2-1237-g646b8c7d-1jammy is installed. 2024-04-22T05:49:45.879 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-22T05:49:46.014 DEBUG:teuthology.orchestra.run.smithi062:> dpkg-query -W -f '${Version}' ceph 2024-04-22T05:49:46.034 INFO:teuthology.orchestra.run.smithi062.stdout:18.2.2-1237-g646b8c7d-1jammy 2024-04-22T05:49:46.034 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1237-g646b8c7d-1jammy 2024-04-22T05:49:46.034 INFO:teuthology.task.install:The correct ceph version 18.2.2-1237-g646b8c7d-1jammy is installed. 2024-04-22T05:49:46.036 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-22T05:49:46.036 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:49:46.036 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T05:49:46.051 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:49:46.051 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T05:49:46.090 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-22T05:49:46.091 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:49:46.091 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T05:49:46.107 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T05:49:46.162 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:49:46.162 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T05:49:46.175 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T05:49:46.228 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-22T05:49:46.229 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:49:46.229 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T05:49:46.242 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T05:49:46.298 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:49:46.298 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T05:49:46.310 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T05:49:46.365 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-22T05:49:46.366 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:49:46.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T05:49:46.379 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T05:49:46.434 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:49:46.434 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T05:49:46.448 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T05:49:46.502 INFO:teuthology.run_tasks:Running task ceph... 2024-04-22T05:49:46.596 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-22T05:49:46.596 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2024-04-22T05:49:46.598 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /var/log/ceph 2024-04-22T05:49:46.612 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-22T05:49:46.612 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-22T05:49:46.656 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-22T05:49:46.669 INFO:tasks.ceph:Creating extra log directories... 2024-04-22T05:49:46.669 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-22T05:49:46.712 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-22T05:49:46.737 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-22T05:49:46.737 INFO:tasks.ceph:config {'conf': {'client': {'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 1500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-22T05:49:46.738 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776', 'branch': 'reef', 'description': 'rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7667776', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 1500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled'], 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'c3a345d31ea77c01771428ab80e1cf32c9b05875'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 931, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 7353, 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d', 'sleep_before_teardown': 0, 'subset': '3333/100000', 'suite': 'rados', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_c3a345d31ea77c01771428ab80e1cf32c9b05875/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'c3a345d31ea77c01771428ab80e1cf32c9b05875', 'targets': {'smithi040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBDc0EB6DtneaHuiYu9Y1eM3NOer0L1aCLtyVQGlyCL8v3dFJlp8ejgZD8QcxGvUFycLqv3OBt4amm0SQ7XLJPM=', 'smithi062.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI82nsOcFRNQibKSdwEJYJV0XxWtfkzblKfQ8RBRWWT1NjFKhQ5xDBnL8q9kSPxxvQ+ptRhLtBiCYCFDu+W7GMI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'}}, {'ceph': None}, {'exec': {'client.0': ['ceph osd pool create cephsqlite', "ceph auth get-or-create client.libcephsqlite mon 'profile simple-rados-client-with-blocklist' osd 'allow rwx pool=cephsqlite' >> /etc/ceph/ceph.keyring"]}}, {'exec': {'client.0': ['ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr']}}, {'workunit': {'clients': {'client.0': ['rados/test_libcephsqlite.sh cephsqlite']}, 'env': {'CEPH_ARGS': '--id libcephsqlite --no-log-to-stderr'}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-21_22:00:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-22T05:49:46.738 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-22T05:49:46.768 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-22T05:49:46.774 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2024-04-22T05:49:46.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/run/ceph 2024-04-22T05:49:46.830 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:49:46.830 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T05:49:46.875 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T05:49:46.875 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-04-22T05:49:46.926 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T05:49:46.926 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T05:49:46.926 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 877 Links: 1 2024-04-22T05:49:46.926 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T05:49:46.926 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-22 05:44:26.995586887 +0000 2024-04-22T05:49:46.926 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-22 05:44:26.747589092 +0000 2024-04-22T05:49:46.926 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-22 05:44:26.747589092 +0000 2024-04-22T05:49:46.926 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-04-22T05:49:46.926 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T05:49:46.981 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-22T05:49:46.981 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-22T05:49:46.982 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000389907 s, 1.3 MB/s 2024-04-22T05:49:46.983 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T05:49:47.033 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-04-22T05:49:47.083 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T05:49:47.083 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T05:49:47.083 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 894 Links: 1 2024-04-22T05:49:47.083 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T05:49:47.083 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-22 05:44:27.495582444 +0000 2024-04-22T05:49:47.083 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-22 05:44:27.223584861 +0000 2024-04-22T05:49:47.083 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-22 05:44:27.223584861 +0000 2024-04-22T05:49:47.083 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-04-22T05:49:47.084 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T05:49:47.137 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-22T05:49:47.138 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-22T05:49:47.138 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000392633 s, 1.3 MB/s 2024-04-22T05:49:47.139 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T05:49:47.188 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-04-22T05:49:47.235 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T05:49:47.235 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T05:49:47.235 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 910 Links: 1 2024-04-22T05:49:47.235 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T05:49:47.235 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-22 05:44:28.019577786 +0000 2024-04-22T05:49:47.236 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-22 05:44:27.743580239 +0000 2024-04-22T05:49:47.236 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-22 05:44:27.743580239 +0000 2024-04-22T05:49:47.236 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-04-22T05:49:47.236 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T05:49:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-22T05:49:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-22T05:49:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000439751 s, 1.2 MB/s 2024-04-22T05:49:47.292 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T05:49:47.340 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-04-22T05:49:47.387 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T05:49:47.387 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T05:49:47.387 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-04-22T05:49:47.388 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T05:49:47.388 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-22 05:44:28.283575439 +0000 2024-04-22T05:49:47.388 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-22 05:44:28.279575474 +0000 2024-04-22T05:49:47.388 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-22 05:44:28.279575474 +0000 2024-04-22T05:49:47.388 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-04-22T05:49:47.388 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T05:49:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-22T05:49:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-22T05:49:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000392714 s, 1.3 MB/s 2024-04-22T05:49:47.443 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T05:49:47.492 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T05:49:47.492 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:49:47.493 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T05:49:47.499 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T05:49:47.499 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-04-22T05:49:47.547 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T05:49:47.547 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T05:49:47.547 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 877 Links: 1 2024-04-22T05:49:47.548 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T05:49:47.548 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-22 05:44:28.034938447 +0000 2024-04-22T05:49:47.548 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-22 05:44:27.770940805 +0000 2024-04-22T05:49:47.548 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-22 05:44:27.770940805 +0000 2024-04-22T05:49:47.548 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2024-04-22T05:49:47.548 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T05:49:47.601 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-22T05:49:47.601 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-22T05:49:47.601 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.00040302 s, 1.3 MB/s 2024-04-22T05:49:47.603 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T05:49:47.656 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-04-22T05:49:47.702 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T05:49:47.702 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T05:49:47.702 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 892 Links: 1 2024-04-22T05:49:47.703 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T05:49:47.703 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-22 05:44:28.530934016 +0000 2024-04-22T05:49:47.703 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-22 05:44:28.274936303 +0000 2024-04-22T05:49:47.703 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-22 05:44:28.274936303 +0000 2024-04-22T05:49:47.703 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2024-04-22T05:49:47.703 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T05:49:47.758 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-22T05:49:47.758 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-22T05:49:47.758 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.00031104 s, 1.6 MB/s 2024-04-22T05:49:47.759 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T05:49:47.807 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-04-22T05:49:47.854 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T05:49:47.854 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T05:49:47.854 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 908 Links: 1 2024-04-22T05:49:47.854 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T05:49:47.854 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-22 05:44:29.062929264 +0000 2024-04-22T05:49:47.855 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-22 05:44:28.818931444 +0000 2024-04-22T05:49:47.855 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-22 05:44:28.818931444 +0000 2024-04-22T05:49:47.855 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2024-04-22T05:49:47.855 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T05:49:47.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-22T05:49:47.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-22T05:49:47.908 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000410171 s, 1.2 MB/s 2024-04-22T05:49:47.909 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T05:49:47.959 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-04-22T05:49:48.006 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T05:49:48.007 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T05:49:48.007 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 928 Links: 1 2024-04-22T05:49:48.007 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T05:49:48.007 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-22 05:44:29.574924690 +0000 2024-04-22T05:49:48.007 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-22 05:44:29.302927121 +0000 2024-04-22T05:49:48.007 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-22 05:44:29.302927121 +0000 2024-04-22T05:49:48.007 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2024-04-22T05:49:48.007 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T05:49:48.061 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-22T05:49:48.061 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-22T05:49:48.061 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000377908 s, 1.4 MB/s 2024-04-22T05:49:48.062 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T05:49:48.111 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T05:49:48.111 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-22T05:49:48.111 INFO:tasks.ceph:Generating config... 2024-04-22T05:49:48.113 INFO:tasks.ceph:[client] debug cephsqlite = 20 2024-04-22T05:49:48.113 INFO:tasks.ceph:[client] debug client = 20 2024-04-22T05:49:48.113 INFO:tasks.ceph:[client] debug ms = 1 2024-04-22T05:49:48.113 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-22T05:49:48.113 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-04-22T05:49:48.113 INFO:tasks.ceph:[global] ms inject socket failures = 1500 2024-04-22T05:49:48.113 INFO:tasks.ceph:[global] ms type = async 2024-04-22T05:49:48.113 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-22T05:49:48.113 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-22T05:49:48.113 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-22T05:49:48.113 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-22T05:49:48.113 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-22T05:49:48.113 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-04-22T05:49:48.113 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-04-22T05:49:48.114 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-04-22T05:49:48.114 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-04-22T05:49:48.114 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-04-22T05:49:48.114 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-04-22T05:49:48.114 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-22T05:49:48.114 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-04-22T05:49:48.114 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-22T05:49:48.114 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-22T05:49:48.114 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-04-22T05:49:48.114 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-22T05:49:48.114 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-22T05:49:48.114 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-22T05:49:48.114 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-22T05:49:48.114 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-22T05:49:48.115 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-22T05:49:48.115 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-22T05:49:48.115 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-22T05:49:48.115 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-04-22T05:49:48.115 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-04-22T05:49:48.115 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-22T05:49:48.115 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-04-22T05:49:48.115 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-04-22T05:49:48.115 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-22T05:49:48.115 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-04-22T05:49:48.115 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-04-22T05:49:48.115 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-22T05:49:48.115 INFO:tasks.ceph:Setting up mon.a... 2024-04-22T05:49:48.116 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-22T05:49:48.154 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2024-04-22T05:49:48.160 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-22T05:49:48.240 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T05:49:48.294 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40'), ('mon.c', '[v2:172.21.15.40:3301,v1:172.21.15.40:6790]'), ('mon.b', '172.21.15.62')] 2024-04-22T05:49:48.294 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.40,[v2:172.21.15.40:3301,v1:172.21.15.40:6790],172.21.15.62', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 1500, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-04-22T05:49:48.295 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:49:48.295 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-22T05:49:48.343 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.40 --addv c '[v2:172.21.15.40:3301,v1:172.21.15.40:6790]' --add b 172.21.15.62 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:49:48.431 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:49:48.431 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid 36e9f223-2858-4559-8f74-0be5d30b8b0d 2024-04-22T05:49:48.431 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = pacific 2024-04-22T05:49:48.431 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2024-04-22T05:49:48.431 INFO:teuthology.orchestra.run.smithi040.stdout:fsid 36e9f223-2858-4559-8f74-0be5d30b8b0d 2024-04-22T05:49:48.431 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2024-04-22T05:49:48.433466+0000 2024-04-22T05:49:48.432 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-04-22T05:49:48.433466+0000 2024-04-22T05:49:48.432 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 16 (pacific) 2024-04-22T05:49:48.432 INFO:teuthology.orchestra.run.smithi040.stdout:election_strategy: 3 2024-04-22T05:49:48.432 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-04-22T05:49:48.432 INFO:teuthology.orchestra.run.smithi040.stdout:1: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.b 2024-04-22T05:49:48.432 INFO:teuthology.orchestra.run.smithi040.stdout:2: [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] mon.c 2024-04-22T05:49:48.432 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-22T05:49:48.433 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-22T05:49:48.438 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 36e9f223-2858-4559-8f74-0be5d30b8b0d... 2024-04-22T05:49:48.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-22T05:49:48.484 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-22T05:49:48.514 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2024-04-22T05:49:48.514 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2024-04-22T05:49:48.514 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-22T05:49:48.514 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = cephx 2024-04-22T05:49:48.515 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.515 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2024-04-22T05:49:48.515 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.515 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2024-04-22T05:49:48.515 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.515 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2024-04-22T05:49:48.515 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2024-04-22T05:49:48.516 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.516 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2024-04-22T05:49:48.516 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2024-04-22T05:49:48.516 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.516 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-22T05:49:48.516 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2024-04-22T05:49:48.516 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.516 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2024-04-22T05:49:48.517 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-22T05:49:48.517 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.517 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2024-04-22T05:49:48.517 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.517 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2024-04-22T05:49:48.517 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2024-04-22T05:49:48.518 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2024-04-22T05:49:48.518 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2024-04-22T05:49:48.518 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2024-04-22T05:49:48.518 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2024-04-22T05:49:48.518 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2024-04-22T05:49:48.518 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-22T05:49:48.518 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-22T05:49:48.519 INFO:teuthology.orchestra.run.smithi040.stdout: mon_allow_pool_size_one = true 2024-04-22T05:49:48.519 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.519 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-22T05:49:48.519 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.519 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2024-04-22T05:49:48.519 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.520 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2024-04-22T05:49:48.520 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.520 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2024-04-22T05:49:48.520 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2024-04-22T05:49:48.520 INFO:teuthology.orchestra.run.smithi040.stdout: mon health detail to clog = false 2024-04-22T05:49:48.520 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = "172.21.15.40,[v2:172.21.15.40:3301,v1:172.21.15.40:6790],172.21.15.62" 2024-04-22T05:49:48.520 INFO:teuthology.orchestra.run.smithi040.stdout: mon client directed command retry = 5 2024-04-22T05:49:48.520 INFO:teuthology.orchestra.run.smithi040.stdout: mon election default strategy = 3 2024-04-22T05:49:48.520 INFO:teuthology.orchestra.run.smithi040.stdout: ms inject socket failures = 1500 2024-04-22T05:49:48.520 INFO:teuthology.orchestra.run.smithi040.stdout: ms type = async 2024-04-22T05:49:48.520 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = 36e9f223-2858-4559-8f74-0be5d30b8b0d 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 600 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock profile = high_recovery_ops 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2024-04-22T05:49:48.521 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.522 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2024-04-22T05:49:48.522 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2024-04-22T05:49:48.522 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.522 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2024-04-22T05:49:48.522 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.522 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2024-04-22T05:49:48.522 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.522 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2024-04-22T05:49:48.522 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.523 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2024-04-22T05:49:48.523 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2024-04-22T05:49:48.523 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.523 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2024-04-22T05:49:48.523 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2024-04-22T05:49:48.523 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2024-04-22T05:49:48.523 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore compression algorithm = zstd 2024-04-22T05:49:48.523 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore compression mode = aggressive 2024-04-22T05:49:48.523 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2024-04-22T05:49:48.523 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore zero block detection = True 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 20 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 20 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 20 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 10 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify cached snaps = True 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify missing on start = True 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 0.95 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock override recovery settings = True 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = bluestore 2024-04-22T05:49:48.524 INFO:teuthology.orchestra.run.smithi040.stdout: osd op queue = debug_random 2024-04-22T05:49:48.525 INFO:teuthology.orchestra.run.smithi040.stdout: osd op queue cut off = debug_random 2024-04-22T05:49:48.525 INFO:teuthology.orchestra.run.smithi040.stdout: osd shutdown pgref assert = True 2024-04-22T05:49:48.525 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.525 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2024-04-22T05:49:48.525 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-22T05:49:48.525 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2024-04-22T05:49:48.525 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-04-22T05:49:48.525 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-04-22T05:49:48.525 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-04-22T05:49:48.525 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-04-22T05:49:48.525 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 20 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.526 INFO:teuthology.orchestra.run.smithi040.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: auth mon ticket ttl = 660 # 11m 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: auth service ticket ttl = 240 # 4m 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: # don't complain about insecure global_id in the test suite 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: # 1m isn't quite enough 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: mon_down_mkfs_grace = 2m 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_filestore_osds = false 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: mon min osdmap epochs = 50 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: mon osdmap full prune interval = 2 2024-04-22T05:49:48.527 INFO:teuthology.orchestra.run.smithi040.stdout: mon osdmap full prune min = 15 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout: mon osdmap full prune txsize = 2 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout: mon scrub interval = 300 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout: paxos service trim min = 10 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout: debug cephsqlite = 20 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2024-04-22T05:49:48.528 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.c] 2024-04-22T05:49:48.529 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.b] 2024-04-22T05:49:48.530 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2024-04-22T05:49:48.530 INFO:teuthology.orchestra.run.smithi062.stdout: chdir = "" 2024-04-22T05:49:48.530 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-22T05:49:48.530 INFO:teuthology.orchestra.run.smithi062.stdout: auth supported = cephx 2024-04-22T05:49:48.530 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.530 INFO:teuthology.orchestra.run.smithi062.stdout: filestore xattr use omap = true 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: mon clock drift allowed = 1.000 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: osd crush chooseleaf type = 0 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: auth debug = true 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on old message = true 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on bug = true 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: mon pg warn max object skew = 0 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: # disable pg_autoscaler by default for new pools 2024-04-22T05:49:48.531 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default size = 2 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow primary affinity = true 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow pg remap = true 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on legacy crush tunables = false 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on crush straw calc version zero = false 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on no sortbitwise = false 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on osd down out interval zero = false 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on too few osds = false 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: mon_allow_pool_size_one = true 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.532 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: osd default data pool replay window = 5 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: mon allow pool delete = true 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: mon cluster log file level = debug 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: debug asserts on shutdown = true 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: mon health detail to clog = false 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: mon host = "172.21.15.40,[v2:172.21.15.40:3301,v1:172.21.15.40:6790],172.21.15.62" 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: mon client directed command retry = 5 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: mon election default strategy = 3 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: ms inject socket failures = 1500 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: ms type = async 2024-04-22T05:49:48.533 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = 36e9f223-2858-4559-8f74-0be5d30b8b0d 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout:[osd] 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout: osd journal size = 100 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub load threshold = 5.0 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub max interval = 600 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout: osd mclock profile = high_recovery_ops 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout: osd recover clone overlap = true 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout: osd recovery max chunk = 1048576 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug shutdown = true 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug op order = true 2024-04-22T05:49:48.534 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify stray on activate = true 2024-04-22T05:49:48.535 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.535 INFO:teuthology.orchestra.run.smithi062.stdout: osd open classes on start = true 2024-04-22T05:49:48.535 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug pg log writeout = true 2024-04-22T05:49:48.535 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.535 INFO:teuthology.orchestra.run.smithi062.stdout: osd deep scrub update digest min age = 30 2024-04-22T05:49:48.535 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.535 INFO:teuthology.orchestra.run.smithi062.stdout: osd map max advance = 10 2024-04-22T05:49:48.535 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.535 INFO:teuthology.orchestra.run.smithi062.stdout: journal zero on create = true 2024-04-22T05:49:48.536 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.536 INFO:teuthology.orchestra.run.smithi062.stdout: filestore ondisk finisher threads = 3 2024-04-22T05:49:48.536 INFO:teuthology.orchestra.run.smithi062.stdout: filestore apply finisher threads = 3 2024-04-22T05:49:48.536 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.536 INFO:teuthology.orchestra.run.smithi062.stdout: bdev debug aio = true 2024-04-22T05:49:48.536 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug misdirected ops = true 2024-04-22T05:49:48.536 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore block size = 96636764160 2024-04-22T05:49:48.536 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore compression algorithm = zstd 2024-04-22T05:49:48.536 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore compression mode = aggressive 2024-04-22T05:49:48.537 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore fsck on mount = True 2024-04-22T05:49:48.537 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore zero block detection = True 2024-04-22T05:49:48.537 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluefs = 20 2024-04-22T05:49:48.537 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluestore = 20 2024-04-22T05:49:48.537 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-22T05:49:48.537 INFO:teuthology.orchestra.run.smithi062.stdout: debug osd = 20 2024-04-22T05:49:48.537 INFO:teuthology.orchestra.run.smithi062.stdout: debug rocksdb = 10 2024-04-22T05:49:48.537 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd backfillfull_ratio = 0.85 2024-04-22T05:49:48.538 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd full ratio = 0.9 2024-04-22T05:49:48.538 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd nearfull ratio = 0.8 2024-04-22T05:49:48.538 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify cached snaps = True 2024-04-22T05:49:48.538 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify missing on start = True 2024-04-22T05:49:48.538 INFO:teuthology.orchestra.run.smithi062.stdout: osd failsafe full ratio = 0.95 2024-04-22T05:49:48.538 INFO:teuthology.orchestra.run.smithi062.stdout: osd mclock override recovery settings = True 2024-04-22T05:49:48.538 INFO:teuthology.orchestra.run.smithi062.stdout: osd objectstore = bluestore 2024-04-22T05:49:48.538 INFO:teuthology.orchestra.run.smithi062.stdout: osd op queue = debug_random 2024-04-22T05:49:48.538 INFO:teuthology.orchestra.run.smithi062.stdout: osd op queue cut off = debug_random 2024-04-22T05:49:48.539 INFO:teuthology.orchestra.run.smithi062.stdout: osd shutdown pgref assert = True 2024-04-22T05:49:48.539 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.539 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr] 2024-04-22T05:49:48.539 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-22T05:49:48.539 INFO:teuthology.orchestra.run.smithi062.stdout: debug mgr = 20 2024-04-22T05:49:48.539 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2024-04-22T05:49:48.539 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2024-04-22T05:49:48.539 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2024-04-22T05:49:48.540 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2024-04-22T05:49:48.540 INFO:teuthology.orchestra.run.smithi062.stdout: mgr/telemetry/nag = false 2024-04-22T05:49:48.540 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.540 INFO:teuthology.orchestra.run.smithi062.stdout:[mon] 2024-04-22T05:49:48.540 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-22T05:49:48.540 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2024-04-22T05:49:48.540 INFO:teuthology.orchestra.run.smithi062.stdout: debug paxos = 20 2024-04-22T05:49:48.540 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2024-04-22T05:49:48.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon data avail warn = 5 2024-04-22T05:49:48.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon mgr mkfs grace = 240 2024-04-22T05:49:48.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2024-04-22T05:49:48.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd reporter subtree level = osd 2024-04-22T05:49:48.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd prime pg temp = true 2024-04-22T05:49:48.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2024-04-22T05:49:48.541 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.541 INFO:teuthology.orchestra.run.smithi062.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-22T05:49:48.541 INFO:teuthology.orchestra.run.smithi062.stdout: auth mon ticket ttl = 660 # 11m 2024-04-22T05:49:48.542 INFO:teuthology.orchestra.run.smithi062.stdout: auth service ticket ttl = 240 # 4m 2024-04-22T05:49:48.542 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.542 INFO:teuthology.orchestra.run.smithi062.stdout: # don't complain about insecure global_id in the test suite 2024-04-22T05:49:48.542 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-22T05:49:48.542 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-22T05:49:48.542 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.542 INFO:teuthology.orchestra.run.smithi062.stdout: # 1m isn't quite enough 2024-04-22T05:49:48.542 INFO:teuthology.orchestra.run.smithi062.stdout: mon_down_mkfs_grace = 2m 2024-04-22T05:49:48.543 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.543 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_filestore_osds = false 2024-04-22T05:49:48.543 INFO:teuthology.orchestra.run.smithi062.stdout: mon min osdmap epochs = 50 2024-04-22T05:49:48.543 INFO:teuthology.orchestra.run.smithi062.stdout: mon osdmap full prune interval = 2 2024-04-22T05:49:48.543 INFO:teuthology.orchestra.run.smithi062.stdout: mon osdmap full prune min = 15 2024-04-22T05:49:48.543 INFO:teuthology.orchestra.run.smithi062.stdout: mon osdmap full prune txsize = 2 2024-04-22T05:49:48.543 INFO:teuthology.orchestra.run.smithi062.stdout: mon scrub interval = 300 2024-04-22T05:49:48.543 INFO:teuthology.orchestra.run.smithi062.stdout: paxos service trim min = 10 2024-04-22T05:49:48.543 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-22T05:49:48.544 INFO:teuthology.orchestra.run.smithi062.stdout:[client] 2024-04-22T05:49:48.544 INFO:teuthology.orchestra.run.smithi062.stdout: rgw cache enabled = true 2024-04-22T05:49:48.544 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable ops log = true 2024-04-22T05:49:48.544 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable usage log = true 2024-04-22T05:49:48.544 INFO:teuthology.orchestra.run.smithi062.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-22T05:49:48.544 INFO:teuthology.orchestra.run.smithi062.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-22T05:49:48.544 INFO:teuthology.orchestra.run.smithi062.stdout: debug cephsqlite = 20 2024-04-22T05:49:48.544 INFO:teuthology.orchestra.run.smithi062.stdout: debug client = 20 2024-04-22T05:49:48.544 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-22T05:49:48.545 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.a] 2024-04-22T05:49:48.545 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.c] 2024-04-22T05:49:48.545 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.b] 2024-04-22T05:49:48.545 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-22T05:49:48.545 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-22T05:49:48.624 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-22T05:49:48.624 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:49:48.624 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-22T05:49:48.630 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:49:48.630 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-22T05:49:48.679 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2024-04-22T05:49:48.679 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:49:48.679 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-22T05:49:48.679 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T05:49:48.746 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:49:48.746 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:49:48.795 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi062.front.sepia.ceph.com 2024-04-22T05:49:48.795 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:49:48.795 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-22T05:49:48.795 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T05:49:48.819 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:49:48.819 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:49:48.867 INFO:tasks.ceph:Setting up mon nodes... 2024-04-22T05:49:48.867 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-22T05:49:48.867 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-22T05:49:48.913 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-22T05:49:48.916 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-22T05:49:48.961 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-22T05:49:48.977 INFO:tasks.ceph:Setting up mds nodes... 2024-04-22T05:49:48.977 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-22T05:49:48.977 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-22T05:49:49.008 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-22T05:49:49.018 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-22T05:49:49.056 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-22T05:49:49.067 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-22T05:49:49.067 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-22T05:49:49.068 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-22T05:49:49.079 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T05:49:49.079 INFO:tasks.ceph:role: osd.0 2024-04-22T05:49:49.079 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-22T05:49:49.080 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-22T05:49:49.142 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T05:49:49.143 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T05:49:49.143 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T05:49:49.143 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-22T05:49:49.143 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T05:49:49.143 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-22T05:49:49.143 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T05:49:49.143 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-22T05:49:49.143 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T05:49:49.143 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T05:49:49.594 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-22T05:49:49.595 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-22T05:49:49.596 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-22T05:49:49.692 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-22T05:49:49.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T05:49:49.702 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2024-04-22T05:49:49.702 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-22T05:49:49.756 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T05:49:49.756 INFO:tasks.ceph:role: osd.1 2024-04-22T05:49:49.756 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-22T05:49:49.756 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-22T05:49:49.819 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T05:49:49.820 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T05:49:49.820 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T05:49:49.820 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-22T05:49:49.820 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T05:49:49.820 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-22T05:49:49.820 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T05:49:49.820 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-22T05:49:49.820 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T05:49:49.820 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T05:49:50.282 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-22T05:49:50.285 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-22T05:49:50.285 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-22T05:49:50.423 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-22T05:49:50.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T05:49:50.435 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2024-04-22T05:49:50.436 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-22T05:49:50.495 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T05:49:50.495 INFO:tasks.ceph:role: osd.2 2024-04-22T05:49:50.495 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-22T05:49:50.495 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-22T05:49:50.558 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T05:49:50.558 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T05:49:50.558 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T05:49:50.558 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-22T05:49:50.559 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T05:49:50.559 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-22T05:49:50.559 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T05:49:50.559 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-22T05:49:50.559 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T05:49:50.559 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T05:49:51.038 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-22T05:49:51.040 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-22T05:49:51.041 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-22T05:49:51.183 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-22T05:49:51.194 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2024-04-22T05:49:51.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T05:49:51.195 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-22T05:49:51.252 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T05:49:51.252 INFO:tasks.ceph:role: osd.3 2024-04-22T05:49:51.252 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-22T05:49:51.252 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-22T05:49:51.315 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T05:49:51.315 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T05:49:51.315 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T05:49:51.315 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-22T05:49:51.315 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T05:49:51.315 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-22T05:49:51.315 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T05:49:51.315 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-22T05:49:51.315 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T05:49:51.315 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T05:49:51.739 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-22T05:49:51.740 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-22T05:49:51.741 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-22T05:49:51.817 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-22T05:49:51.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T05:49:51.868 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2024-04-22T05:49:51.869 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:49:51.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:51.978+0000 7fd9b05827c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-22T05:49:51.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:51.986+0000 7fd9b05827c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-22T05:49:51.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:51.986+0000 7fd9b05827c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T05:49:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:51.986+0000 7fd9b05827c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T05:49:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:51.986+0000 7fd9b05827c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T05:49:51.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:51.990+0000 7fd9b05827c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-22T05:49:54.991 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-22T05:49:55.006 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:49:55.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:55.086+0000 7feb6929f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-22T05:49:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:55.090+0000 7feb6929f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-22T05:49:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:55.090+0000 7feb6929f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T05:49:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:55.090+0000 7feb6929f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T05:49:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:55.090+0000 7feb6929f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T05:49:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:55.090+0000 7feb6929f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-22T05:49:58.066 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-22T05:49:58.120 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:49:58.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:58.202+0000 7fc764d127c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-22T05:49:58.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:58.202+0000 7fc764d127c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-22T05:49:58.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:58.202+0000 7fc764d127c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T05:49:58.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:58.202+0000 7fc764d127c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T05:49:58.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:58.202+0000 7fc764d127c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T05:49:58.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:49:58.202+0000 7fc764d127c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-22T05:50:01.182 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-22T05:50:01.243 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:50:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:01.326+0000 7f6a3953b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-22T05:50:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:01.326+0000 7f6a3953b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-22T05:50:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:01.326+0000 7f6a3953b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T05:50:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:01.326+0000 7f6a3953b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T05:50:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:01.326+0000 7f6a3953b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T05:50:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:01.326+0000 7f6a3953b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-22T05:50:04.290 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-22T05:50:04.347 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-22T05:50:04.362 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T05:50:04.362 INFO:tasks.ceph:role: osd.4 2024-04-22T05:50:04.362 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-22T05:50:04.362 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-22T05:50:04.425 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T05:50:04.425 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T05:50:04.425 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T05:50:04.426 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-22T05:50:04.426 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T05:50:04.426 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-22T05:50:04.426 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T05:50:04.426 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-22T05:50:04.426 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T05:50:04.426 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T05:50:04.885 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-22T05:50:04.887 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-22T05:50:04.887 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-22T05:50:04.980 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-22T05:50:04.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T05:50:04.992 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: /sbin/restorecon: command not found 2024-04-22T05:50:04.992 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-22T05:50:05.045 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T05:50:05.045 INFO:tasks.ceph:role: osd.5 2024-04-22T05:50:05.045 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-22T05:50:05.045 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-22T05:50:05.113 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T05:50:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T05:50:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T05:50:05.113 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-22T05:50:05.114 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T05:50:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-22T05:50:05.114 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T05:50:05.114 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-22T05:50:05.114 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T05:50:05.114 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T05:50:05.581 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-22T05:50:05.583 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-22T05:50:05.583 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-22T05:50:05.723 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-22T05:50:05.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T05:50:05.736 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: /sbin/restorecon: command not found 2024-04-22T05:50:05.737 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-22T05:50:05.790 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T05:50:05.790 INFO:tasks.ceph:role: osd.6 2024-04-22T05:50:05.790 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-22T05:50:05.790 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-22T05:50:05.853 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T05:50:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T05:50:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T05:50:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-22T05:50:05.853 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T05:50:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-22T05:50:05.853 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T05:50:05.853 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-22T05:50:05.853 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T05:50:05.853 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T05:50:06.334 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-22T05:50:06.336 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-22T05:50:06.337 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-22T05:50:06.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-22T05:50:06.480 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: /sbin/restorecon: command not found 2024-04-22T05:50:06.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T05:50:06.481 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-22T05:50:06.537 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T05:50:06.538 INFO:tasks.ceph:role: osd.7 2024-04-22T05:50:06.538 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-22T05:50:06.538 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-22T05:50:06.602 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T05:50:06.603 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T05:50:06.603 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T05:50:06.603 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-22T05:50:06.603 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T05:50:06.603 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-22T05:50:06.603 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T05:50:06.603 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-22T05:50:06.603 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T05:50:06.603 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T05:50:07.049 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-22T05:50:07.051 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-22T05:50:07.051 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-22T05:50:07.141 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-22T05:50:07.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T05:50:07.153 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: /sbin/restorecon: command not found 2024-04-22T05:50:07.153 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:50:07.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:07.274+0000 7fc5019917c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-22T05:50:07.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:07.274+0000 7fc5019917c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-22T05:50:07.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:07.274+0000 7fc5019917c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T05:50:07.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:07.274+0000 7fc5019917c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T05:50:07.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:07.274+0000 7fc5019917c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T05:50:07.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:07.278+0000 7fc5019917c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-22T05:50:10.279 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-22T05:50:10.293 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:50:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:10.378+0000 7ff914eb27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-22T05:50:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:10.382+0000 7ff914eb27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-22T05:50:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:10.382+0000 7ff914eb27c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T05:50:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:10.382+0000 7ff914eb27c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T05:50:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:10.382+0000 7ff914eb27c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T05:50:10.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:10.382+0000 7ff914eb27c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-22T05:50:13.350 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-22T05:50:13.406 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:50:13.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:13.490+0000 7ffb2d1037c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-22T05:50:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:13.490+0000 7ffb2d1037c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-22T05:50:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:13.490+0000 7ffb2d1037c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T05:50:13.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:13.490+0000 7ffb2d1037c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T05:50:13.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:13.490+0000 7ffb2d1037c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T05:50:13.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:13.490+0000 7ffb2d1037c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-22T05:50:16.481 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-22T05:50:16.539 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:50:16.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:16.622+0000 7f510baf87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-22T05:50:16.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:16.622+0000 7f510baf87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-22T05:50:16.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:16.622+0000 7f510baf87c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T05:50:16.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:16.622+0000 7f510baf87c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T05:50:16.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:16.622+0000 7f510baf87c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T05:50:16.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:16.622+0000 7f510baf87c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-22T05:50:19.621 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-22T05:50:19.673 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-22T05:50:19.674 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:50:19.674 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-22T05:50:19.689 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:50:19.689 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-22T05:50:19.743 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:50:19.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-22T05:50:19.803 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:50:19.803 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-22T05:50:19.858 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:50:19.858 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-22T05:50:19.918 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:50:19.919 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-22T05:50:19.932 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:50:19.932 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-22T05:50:19.985 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:50:19.985 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-22T05:50:20.045 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:50:20.045 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-22T05:50:20.102 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:50:20.102 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-22T05:50:20.158 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:50:20.158 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-22T05:50:20.164 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:50:20.164 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-22T05:50:20.206 INFO:tasks.ceph:Adding keys to all mons... 2024-04-22T05:50:20.207 DEBUG:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-22T05:50:20.209 DEBUG:teuthology.orchestra.run.smithi062:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-22T05:50:20.252 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.y] 2024-04-22T05:50:20.252 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB8+iVm6LyLNhAAPGlSseuPCubShkQzM4Uabw== 2024-04-22T05:50:20.252 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2024-04-22T05:50:20.252 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB/+iVmarinOhAANjZv5WAtU/ki8AvJ8njnJA== 2024-04-22T05:50:20.252 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2024-04-22T05:50:20.252 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCD+iVmO2KFBRAAuiABsLw2B2VLf3pkXt1CTw== 2024-04-22T05:50:20.253 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2024-04-22T05:50:20.253 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCG+iVme7g4DBAAcAZhjVHIYRInXCS/RkVDXQ== 2024-04-22T05:50:20.253 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2024-04-22T05:50:20.253 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCJ+iVmlDWvExAAPBiTfKY3rCLeqQjNuaFT8Q== 2024-04-22T05:50:20.253 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2024-04-22T05:50:20.253 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB8+iVmNzl7ORAAKK+sTk6kwBfP8RTxjewXsQ== 2024-04-22T05:50:20.253 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.4] 2024-04-22T05:50:20.253 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCP+iVm3h6tEBAALCd9S74JsRyi3i69XXvbGA== 2024-04-22T05:50:20.253 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.5] 2024-04-22T05:50:20.253 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCS+iVmh2XsFhAAl5pRVtUYzvUlZzKaFgNBxQ== 2024-04-22T05:50:20.253 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.6] 2024-04-22T05:50:20.253 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCV+iVm6+hmHRAApkBkRFxT0V/XPh04mFK56w== 2024-04-22T05:50:20.253 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.7] 2024-04-22T05:50:20.254 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCY+iVm88JGJRAAzmEPk0crlF4H/9Tc0OtI1g== 2024-04-22T05:50:20.254 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-22T05:50:20.254 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB9+iVmhIOnABAAOrib5QSs+EplBFNgfcsfxQ== 2024-04-22T05:50:20.254 INFO:teuthology.orchestra.run.smithi040.stdout:[client.1] 2024-04-22T05:50:20.254 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB9+iVmGQOOAxAA5NCHg5SqkOLg2uYVifs1OQ== 2024-04-22T05:50:20.260 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.y] 2024-04-22T05:50:20.260 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB8+iVm6LyLNhAAPGlSseuPCubShkQzM4Uabw== 2024-04-22T05:50:20.260 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.0] 2024-04-22T05:50:20.260 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB/+iVmarinOhAANjZv5WAtU/ki8AvJ8njnJA== 2024-04-22T05:50:20.260 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.1] 2024-04-22T05:50:20.260 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCD+iVmO2KFBRAAuiABsLw2B2VLf3pkXt1CTw== 2024-04-22T05:50:20.260 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.2] 2024-04-22T05:50:20.260 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCG+iVme7g4DBAAcAZhjVHIYRInXCS/RkVDXQ== 2024-04-22T05:50:20.260 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.3] 2024-04-22T05:50:20.260 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCJ+iVmlDWvExAAPBiTfKY3rCLeqQjNuaFT8Q== 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.x] 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB8+iVmNzl7ORAAKK+sTk6kwBfP8RTxjewXsQ== 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.4] 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCP+iVm3h6tEBAALCd9S74JsRyi3i69XXvbGA== 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.5] 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCS+iVmh2XsFhAAl5pRVtUYzvUlZzKaFgNBxQ== 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.6] 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCV+iVm6+hmHRAApkBkRFxT0V/XPh04mFK56w== 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.7] 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCY+iVm88JGJRAAzmEPk0crlF4H/9Tc0OtI1g== 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB9+iVmhIOnABAAOrib5QSs+EplBFNgfcsfxQ== 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout:[client.1] 2024-04-22T05:50:20.261 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB9+iVmGQOOAxAA5NCHg5SqkOLg2uYVifs1OQ== 2024-04-22T05:50:20.262 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T05:50:20.300 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T05:50:20.352 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.388 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.472 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.514 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.556 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.640 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.683 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T05:50:20.728 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T05:50:20.772 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.774 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.854 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.863 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.944 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.946 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.992 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:20.994 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T05:50:21.078 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T05:50:21.081 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T05:50:21.127 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T05:50:21.129 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T05:50:21.175 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-22T05:50:21.175 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-22T05:50:21.229 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T05:50:21.320 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-22T05:50:21.331 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-22T05:50:21.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T05:50:21.474 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-22T05:50:21.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-22T05:50:21.540 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T05:50:21.633 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-22T05:50:21.644 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:50:21.646 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-22T05:50:21.689 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-22T05:50:21.689 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-22T05:50:21.689 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-22T05:50:21.695 INFO:tasks.ceph.mon.a:Started 2024-04-22T05:50:21.696 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-22T05:50:21.696 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-22T05:50:21.698 INFO:tasks.ceph.mon.c:Started 2024-04-22T05:50:21.698 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-22T05:50:21.698 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-22T05:50:21.734 INFO:tasks.ceph.mon.b:Started 2024-04-22T05:50:21.734 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-22T05:50:21.734 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-22T05:50:21.735 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-22T05:50:21.737 INFO:tasks.ceph.mgr.y:Started 2024-04-22T05:50:21.737 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-22T05:50:21.737 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-22T05:50:21.739 INFO:tasks.ceph.mgr.x:Started 2024-04-22T05:50:21.739 INFO:tasks.ceph:Setting crush tunables to default 2024-04-22T05:50:21.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2024-04-22T05:50:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.854+0000 7f5715711640 1 Processor -- start 2024-04-22T05:50:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.854+0000 7f5715711640 1 -- start start 2024-04-22T05:50:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.854+0000 7f5715711640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.854+0000 7f5715711640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.854+0000 7f5715711640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5710142820 con 0x7f5710142440 2024-04-22T05:50:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.854+0000 7f5715711640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5710142ae0 con 0x7f571006ff70 2024-04-22T05:50:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.854+0000 7f5715711640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5710070940 con 0x7f57100b0af0 2024-04-22T05:50:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.854+0000 7f570e7fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.854+0000 7f570e7fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:33116/0 (socket says 172.21.15.40:33116) 2024-04-22T05:50:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.854+0000 7f570e7fc640 1 -- 172.21.15.40:0/2543350772 learned_addr learned my addr 172.21.15.40:0/2543350772 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.854+0000 7f570e7fc640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 msgr2=0x7f57100afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T05:50:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.854+0000 7f570e7fc640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 msgr2=0x7f57100afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.854+0000 7f570e7fc640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.858+0000 7f570e7fc640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T05:50:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.858+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.858+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 msgr2=0x7f5710070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-22T05:50:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.858+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 msgr2=0x7f5710070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.858+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:21.858+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T05:50:22.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.062+0000 7f570e7fc640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:22.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.062+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:22.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.062+0000 7f570e7fc640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 msgr2=0x7f57100afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T05:50:22.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.062+0000 7f570e7fc640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 msgr2=0x7f57100afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:22.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.062+0000 7f570e7fc640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:22.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.062+0000 7f570e7fc640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T05:50:22.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.062+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 msgr2=0x7f5710070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-22T05:50:22.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.062+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 msgr2=0x7f5710070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:22.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.062+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:22.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.062+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T05:50:22.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.466+0000 7f570e7fc640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:22.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.466+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:22.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.466+0000 7f570e7fc640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 msgr2=0x7f57100afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-22T05:50:22.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.466+0000 7f570e7fc640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 msgr2=0x7f57100afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:22.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.466+0000 7f570e7fc640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:22.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.466+0000 7f570e7fc640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T05:50:22.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.466+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 msgr2=0x7f5710070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T05:50:22.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.466+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 msgr2=0x7f5710070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:22.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.466+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:22.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:22.466+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T05:50:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:23.270+0000 7f570e7fc640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:23.270+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:23.270+0000 7f570e7fc640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 msgr2=0x7f57100afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-22T05:50:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:23.270+0000 7f570e7fc640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 msgr2=0x7f57100afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:23.270+0000 7f570e7fc640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:23.270+0000 7f570e7fc640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T05:50:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:23.270+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 msgr2=0x7f5710070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T05:50:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:23.270+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 msgr2=0x7f5710070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:23.270+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:23.270+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T05:50:24.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:24.854+0000 7f570cff9640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 msgr2=0x7f57100afcb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:24.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:24.854+0000 7f570cff9640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f57100b0af0 0x7f57100afcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:24.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:24.854+0000 7f570cff9640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 msgr2=0x7f5710070400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:24.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:24.854+0000 7f570cff9640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.62:3300/0 conn(0x7f571006ff70 0x7f5710070400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:24.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:24.854+0000 7f570cff9640 1 -- 172.21.15.40:0/2543350772 >> v1:172.21.15.62:6789/0 conn(0x7f5710142440 legacy=0x7f571013bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:24.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:24.854+0000 7f570cff9640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:24.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:24.854+0000 7f570cff9640 1 -- 172.21.15.40:0/2543350772 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57100b1340 con 0x7f56fc006220 2024-04-22T05:50:24.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:24.854+0000 7f570cff9640 1 -- 172.21.15.40:0/2543350772 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5710142ae0 con 0x7f56fc0036f0 2024-04-22T05:50:24.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:24.858+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:24.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:24.858+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T05:50:24.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:24.858+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:24.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:24.858+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:24.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:24.858+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T05:50:25.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:25.062+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:25.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:25.062+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T05:50:25.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:25.062+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:25.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:25.062+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:25.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:25.062+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T05:50:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:25.466+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:25.466+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T05:50:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:25.466+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:25.466+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:25.466+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T05:50:26.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:26.270+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:26.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:26.270+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T05:50:26.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:26.270+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:26.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:26.270+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:26.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:26.270+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T05:50:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570cff9640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570cff9640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570cff9640 1 -- 172.21.15.40:0/2543350772 >> v1:172.21.15.40:6789/0 conn(0x7f56fc006220 legacy=0x7f56fc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570cff9640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 reap_dead start 2024-04-22T05:50:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570cff9640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 0x7f56fc0086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570cff9640 1 -- 172.21.15.40:0/2543350772 --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56fc00afd0 con 0x7f5710142440 2024-04-22T05:50:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570cff9640 1 -- 172.21.15.40:0/2543350772 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5710142ae0 con 0x7f56fc0036f0 2024-04-22T05:50:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570cff9640 1 -- 172.21.15.40:0/2543350772 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5710070940 con 0x7f56fc00a7c0 2024-04-22T05:50:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 0x7f56fc0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 msgr2=0x7f56fc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-22T05:50:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 msgr2=0x7f56fc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 0x7f56fc0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T05:50:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 0x7f56fc0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T05:50:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:27.858+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T05:50:28.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.062+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 0x7f56fc0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.062+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.062+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 msgr2=0x7f56fc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T05:50:28.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.062+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 msgr2=0x7f56fc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:28.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.062+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 0x7f56fc0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:28.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.062+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-22T05:50:28.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.062+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:28.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.062+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:28.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.062+0000 7f570f7fe640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 0x7f56fc0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T05:50:28.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.062+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T05:50:28.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491545130 0 0) 0x7f56fc00afd0 con 0x7f5710142440 2024-04-22T05:50:28.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56f8003530 con 0x7f5710142440 2024-04-22T05:50:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2534961216 0 0) 0x7f56f8003530 con 0x7f5710142440 2024-04-22T05:50:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 msgr2=0x7f56fc0086e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 0x7f56fc0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 msgr2=0x7f56fc005b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:28.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 --2- 172.21.15.40:0/2543350772 >> v2:172.21.15.40:3300/0 conn(0x7f56fc0036f0 0x7f56fc005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc00abf0 con 0x7f5710142440 2024-04-22T05:50:28.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 <== mon.1 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268782488 0 0) 0x7f570400c570 con 0x7f5710142440 2024-04-22T05:50:28.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 >> v1:172.21.15.62:6789/0 conn(0x7f5710142440 legacy=0x7f56fc00a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56f80039f0 0x7f56f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56f80047d0 0x7f56f8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 0x7f56f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5710142ae0 con 0x7f56f80039f0 2024-04-22T05:50:28.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5710070940 con 0x7f56f80047d0 2024-04-22T05:50:28.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80079f0 con 0x7f56fc00a7c0 2024-04-22T05:50:28.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56f80039f0 0x7f56f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 msgr2=0x7f56f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:28.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f56fc00a7c0 0x7f56f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56f80047d0 msgr2=0x7f56f8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:28.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56f80047d0 0x7f56f8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.114+0000 7f570effd640 1 -- 172.21.15.40:0/2543350772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8003530 con 0x7f56f80039f0 2024-04-22T05:50:28.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.122+0000 7f570effd640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56f80039f0 0x7f56f8003e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5700003600 tx=0x7f570001d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228f84b96e6ac31 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:28.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.122+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700007020 con 0x7f56f80039f0 2024-04-22T05:50:28.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.122+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5700007180 con 0x7f56f80039f0 2024-04-22T05:50:28.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.122+0000 7f570dffb640 1 -- 172.21.15.40:0/2543350772 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700007530 con 0x7f56f80039f0 2024-04-22T05:50:28.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.126+0000 7f5715711640 1 -- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56f80039f0 msgr2=0x7f56f8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.126+0000 7f5715711640 1 --2- 172.21.15.40:0/2543350772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56f80039f0 0x7f56f8003e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5700003600 tx=0x7f570001d7c0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.126+0000 7f570f7fe640 1 -- 172.21.15.40:0/2543350772 reap_dead start 2024-04-22T05:50:28.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.126+0000 7f5715711640 1 -- 172.21.15.40:0/2543350772 shutdown_connections 2024-04-22T05:50:28.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.126+0000 7f5715711640 1 -- 172.21.15.40:0/2543350772 >> 172.21.15.40:0/2543350772 conn(0x7f57100b4af0 msgr2=0x7f57100b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:28.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.126+0000 7f5715711640 1 -- 172.21.15.40:0/2543350772 shutdown_connections 2024-04-22T05:50:28.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.126+0000 7f5715711640 1 -- 172.21.15.40:0/2543350772 wait complete. 2024-04-22T05:50:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.126+0000 7f5715711640 1 Processor -- start 2024-04-22T05:50:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.126+0000 7f5715711640 1 -- start start 2024-04-22T05:50:28.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f5715711640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57101cbb00 0x7f57101cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f5715711640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5710136dc0 0x7f5710137220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f5715711640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57101c8850 0x7f57101c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f5715711640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5710137760 con 0x7f57101c8850 2024-04-22T05:50:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f5715711640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100a8120 con 0x7f57101cbb00 2024-04-22T05:50:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f5715711640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57101c83c0 con 0x7f5710136dc0 2024-04-22T05:50:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f570e7fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5710136dc0 0x7f5710137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f570e7fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5710136dc0 0x7f5710137220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:33240/0 (socket says 172.21.15.40:33240) 2024-04-22T05:50:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f570effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57101cbb00 0x7f57101cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f570e7fc640 1 -- 172.21.15.40:0/3619131246 learned_addr learned my addr 172.21.15.40:0/3619131246 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f570f7fe640 1 --2- 172.21.15.40:0/3619131246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57101c8850 0x7f57101c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f570e7fc640 1 -- 172.21.15.40:0/3619131246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57101cbb00 msgr2=0x7f57101cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f570e7fc640 1 --2- 172.21.15.40:0/3619131246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57101cbb00 0x7f57101cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f570e7fc640 1 -- 172.21.15.40:0/3619131246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57101c8850 msgr2=0x7f57101c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f570e7fc640 1 --2- 172.21.15.40:0/3619131246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57101c8850 0x7f57101c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f570e7fc640 1 -- 172.21.15.40:0/3619131246 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57101ca150 con 0x7f5710136dc0 2024-04-22T05:50:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f570e7fc640 1 --2- 172.21.15.40:0/3619131246 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5710136dc0 0x7f5710137220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5704014910 tx=0x7f5704014dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f56f77fe640 1 -- 172.21.15.40:0/3619131246 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57040129f0 con 0x7f5710136dc0 2024-04-22T05:50:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f56f77fe640 1 -- 172.21.15.40:0/3619131246 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570400c040 con 0x7f5710136dc0 2024-04-22T05:50:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f5715711640 1 -- 172.21.15.40:0/3619131246 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5710137910 con 0x7f5710136dc0 2024-04-22T05:50:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f56f77fe640 1 -- 172.21.15.40:0/3619131246 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704012d90 con 0x7f5710136dc0 2024-04-22T05:50:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f5715711640 1 -- 172.21.15.40:0/3619131246 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57101c93e0 con 0x7f5710136dc0 2024-04-22T05:50:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f56f77fe640 1 -- 172.21.15.40:0/3619131246 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f5704017770 con 0x7f5710136dc0 2024-04-22T05:50:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f56f77fe640 1 -- 172.21.15.40:0/3619131246 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5704022c20 con 0x7f5710136dc0 2024-04-22T05:50:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.130+0000 7f5715711640 1 -- 172.21.15.40:0/3619131246 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5710066d70 con 0x7f5710136dc0 2024-04-22T05:50:28.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.142+0000 7f56f77fe640 1 -- 172.21.15.40:0/3619131246 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f5704027060 con 0x7f5710136dc0 2024-04-22T05:50:28.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.210+0000 7f5715711640 1 -- 172.21.15.40:0/3619131246 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5710137b00 con 0x7f5710136dc0 2024-04-22T05:50:28.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.214+0000 7f56f77fe640 1 -- 172.21.15.40:0/3619131246 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f570401d070 con 0x7f5710136dc0 2024-04-22T05:50:28.216 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2024-04-22T05:50:28.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.214+0000 7f5715711640 1 -- 172.21.15.40:0/3619131246 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5710136dc0 msgr2=0x7f5710137220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.214+0000 7f5715711640 1 --2- 172.21.15.40:0/3619131246 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5710136dc0 0x7f5710137220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5704014910 tx=0x7f5704014dd0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.214+0000 7f5715711640 1 -- 172.21.15.40:0/3619131246 shutdown_connections 2024-04-22T05:50:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.214+0000 7f5715711640 1 --2- 172.21.15.40:0/3619131246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57101c8850 0x7f57101c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.214+0000 7f5715711640 1 --2- 172.21.15.40:0/3619131246 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5710136dc0 0x7f5710137220 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.214+0000 7f5715711640 1 --2- 172.21.15.40:0/3619131246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57101cbb00 0x7f57101cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.214+0000 7f5715711640 1 -- 172.21.15.40:0/3619131246 >> 172.21.15.40:0/3619131246 conn(0x7f57100b4af0 msgr2=0x7f5710070ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.214+0000 7f5715711640 1 -- 172.21.15.40:0/3619131246 shutdown_connections 2024-04-22T05:50:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:28.214+0000 7f5715711640 1 -- 172.21.15.40:0/3619131246 wait complete. 2024-04-22T05:50:28.235 INFO:tasks.ceph:check_enable_crimson: False 2024-04-22T05:50:28.235 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-22T05:50:28.235 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:50:28.235 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-22T05:50:28.246 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:50:28.246 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-22T05:50:28.302 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:50:28.302 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-22T05:50:28.339 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:28.338+0000 7f509a89e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T05:50:28.361 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:50:28.361 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-22T05:50:28.385 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:28.382+0000 7f7ca365d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T05:50:28.394 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:50:28.394 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-22T05:50:28.409 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:50:28.409 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-22T05:50:28.465 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:50:28.465 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-22T05:50:28.521 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-22T05:50:28.521 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-22T05:50:28.577 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 1997ac14-110c-4d7e-8ac5-d0ac4f70af75 0 2024-04-22T05:50:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd9cf0640 1 Processor -- start 2024-04-22T05:50:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd9cf0640 1 -- start start 2024-04-22T05:50:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd9cf0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2cd413c5e0 0x7f2cd40afd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd9cf0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd413f3f0 con 0x7f2cd40b0bb0 2024-04-22T05:50:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd9cf0640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd4070610 con 0x7f2cd40af9b0 2024-04-22T05:50:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd9cf0640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd40a7bb0 con 0x7f2cd413c5e0 2024-04-22T05:50:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd3fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2cd413c5e0 0x7f2cd40afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd37fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2cd40af9b0 0x7f2cd406ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52632/0 (socket says 172.21.15.62:52632) 2024-04-22T05:50:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd3fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2cd413c5e0 0x7f2cd40afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38364/0 (socket says 172.21.15.62:38364) 2024-04-22T05:50:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd37fe640 1 -- 172.21.15.62:0/3218598077 learned_addr learned my addr 172.21.15.62:0/3218598077 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-22T05:50:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd27fc640 1 -- 172.21.15.62:0/3218598077 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026950995 0 0) 0x7f2cd4070610 con 0x7f2cd40af9b0 2024-04-22T05:50:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd27fc640 1 -- 172.21.15.62:0/3218598077 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cbc003530 con 0x7f2cd40af9b0 2024-04-22T05:50:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd3fff640 1 -- 172.21.15.62:0/3218598077 >> v1:172.21.15.62:6789/0 conn(0x7f2cd40af9b0 legacy=0x7f2cd406ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd3fff640 1 -- 172.21.15.62:0/3218598077 >> v1:172.21.15.40:6789/0 conn(0x7f2cd40b0bb0 legacy=0x7f2cd413bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.746+0000 7f2cd3fff640 1 -- 172.21.15.62:0/3218598077 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd413f880 con 0x7f2cd413c5e0 2024-04-22T05:50:28.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.750+0000 7f2cd3fff640 1 --2- 172.21.15.62:0/3218598077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd413c5e0 0x7f2cd40afd90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009d90 tx=0x7f2ccc01c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0b156f1b2c36854 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:28.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.750+0000 7f2cd27fc640 1 -- 172.21.15.62:0/3218598077 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0a5a30 con 0x7f2cd413c5e0 2024-04-22T05:50:28.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.750+0000 7f2cd27fc640 1 -- 172.21.15.62:0/3218598077 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc0a5b90 con 0x7f2cd413c5e0 2024-04-22T05:50:28.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.750+0000 7f2cd27fc640 1 -- 172.21.15.62:0/3218598077 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0ad440 con 0x7f2cd413c5e0 2024-04-22T05:50:28.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.750+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/3218598077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd413c5e0 msgr2=0x7f2cd40afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.750+0000 7f2cd9cf0640 1 --2- 172.21.15.62:0/3218598077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd413c5e0 0x7f2cd40afd90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009d90 tx=0x7f2ccc01c950 comp rx=0 tx=0).stop 2024-04-22T05:50:28.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.750+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/3218598077 shutdown_connections 2024-04-22T05:50:28.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.750+0000 7f2cd9cf0640 1 --2- 172.21.15.62:0/3218598077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd413c5e0 0x7f2cd40afd90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.750+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/3218598077 >> 172.21.15.62:0/3218598077 conn(0x7f2cd40b4cc0 msgr2=0x7f2cd40b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:28.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.750+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/3218598077 shutdown_connections 2024-04-22T05:50:28.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.750+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/3218598077 wait complete. 2024-04-22T05:50:28.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd9cf0640 1 Processor -- start 2024-04-22T05:50:28.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd9cf0640 1 -- start start 2024-04-22T05:50:28.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd9cf0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd40af9b0 0x7f2cd413a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd9cf0640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cd40b0bb0 0x7f2cd41387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd9cf0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd413c5e0 0x7f2cd4132f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd9cf0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4138d10 con 0x7f2cd40af9b0 2024-04-22T05:50:28.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd9cf0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4137170 con 0x7f2cd413c5e0 2024-04-22T05:50:28.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd9cf0640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4138f90 con 0x7f2cd40b0bb0 2024-04-22T05:50:28.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd413c5e0 0x7f2cd4132f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd413c5e0 0x7f2cd4132f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38380/0 (socket says 172.21.15.62:38380) 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd3fff640 1 -- 172.21.15.62:0/1928900332 learned_addr learned my addr 172.21.15.62:0/1928900332 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd37fe640 1 --2- 172.21.15.62:0/1928900332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd40af9b0 0x7f2cd413a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd2ffd640 1 --2- 172.21.15.62:0/1928900332 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cd40b0bb0 0x7f2cd41387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd3fff640 1 -- 172.21.15.62:0/1928900332 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cd40b0bb0 msgr2=0x7f2cd41387d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd3fff640 1 --2- 172.21.15.62:0/1928900332 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cd40b0bb0 0x7f2cd41387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd3fff640 1 -- 172.21.15.62:0/1928900332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd40af9b0 msgr2=0x7f2cd413a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd3fff640 1 --2- 172.21.15.62:0/1928900332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd40af9b0 0x7f2cd413a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd3fff640 1 -- 172.21.15.62:0/1928900332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc009a70 con 0x7f2cd413c5e0 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd37fe640 1 --2- 172.21.15.62:0/1928900332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd40af9b0 0x7f2cd413a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd3fff640 1 --2- 172.21.15.62:0/1928900332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd413c5e0 0x7f2cd4132f20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc01a580 tx=0x7f2ccc0196f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cbbfff640 1 -- 172.21.15.62:0/1928900332 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0a5a30 con 0x7f2cd413c5e0 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cbbfff640 1 -- 172.21.15.62:0/1928900332 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc009450 con 0x7f2cd413c5e0 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/1928900332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd4137d00 con 0x7f2cd413c5e0 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cbbfff640 1 -- 172.21.15.62:0/1928900332 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0b9650 con 0x7f2cd413c5e0 2024-04-22T05:50:28.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/1928900332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd4133630 con 0x7f2cd413c5e0 2024-04-22T05:50:28.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cbbfff640 1 -- 172.21.15.62:0/1928900332 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f2ccc0b97b0 con 0x7f2cd413c5e0 2024-04-22T05:50:28.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cbbfff640 1 -- 172.21.15.62:0/1928900332 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2ccc0c4980 con 0x7f2cd413c5e0 2024-04-22T05:50:28.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.754+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/1928900332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd41391b0 con 0x7f2cd413c5e0 2024-04-22T05:50:28.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.766+0000 7f2cbbfff640 1 -- 172.21.15.62:0/1928900332 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f2ccc0b2070 con 0x7f2cd413c5e0 2024-04-22T05:50:28.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.830+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/1928900332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1997ac14-110c-4d7e-8ac5-d0ac4f70af75", "id": 0} v 0) v1 -- 0x7f2cd41391b0 con 0x7f2cd413c5e0 2024-04-22T05:50:28.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.838+0000 7f2cbbfff640 1 -- 172.21.15.62:0/1928900332 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1997ac14-110c-4d7e-8ac5-d0ac4f70af75", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2ccc0b9b50 con 0x7f2cd413c5e0 2024-04-22T05:50:28.839 INFO:teuthology.orchestra.run.smithi062.stdout:0 2024-04-22T05:50:28.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.838+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/1928900332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd413c5e0 msgr2=0x7f2cd4132f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.838+0000 7f2cd9cf0640 1 --2- 172.21.15.62:0/1928900332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd413c5e0 0x7f2cd4132f20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc01a580 tx=0x7f2ccc0196f0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.838+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/1928900332 shutdown_connections 2024-04-22T05:50:28.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.838+0000 7f2cd9cf0640 1 --2- 172.21.15.62:0/1928900332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd413c5e0 0x7f2cd4132f20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.838+0000 7f2cd9cf0640 1 --2- 172.21.15.62:0/1928900332 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2cd40b0bb0 0x7f2cd41387d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.838+0000 7f2cd9cf0640 1 --2- 172.21.15.62:0/1928900332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd40af9b0 0x7f2cd413a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.838+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/1928900332 >> 172.21.15.62:0/1928900332 conn(0x7f2cd40b4cc0 msgr2=0x7f2cd4065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:28.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.838+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/1928900332 shutdown_connections 2024-04-22T05:50:28.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.838+0000 7f2cd9cf0640 1 -- 172.21.15.62:0/1928900332 wait complete. 2024-04-22T05:50:28.856 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new cb6b2995-f7ce-4a68-ac16-d1690cc2b9ac 1 2024-04-22T05:50:28.860 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:28.858+0000 7f509a89e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T05:50:28.939 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:28.938+0000 7f7ca365d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T05:50:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.974+0000 7f62870de640 1 Processor -- start 2024-04-22T05:50:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.974+0000 7f62870de640 1 -- start start 2024-04-22T05:50:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.974+0000 7f62870de640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f628006f800 0x7f62800ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.974+0000 7f62870de640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628006fbe0 con 0x7f628013aad0 2024-04-22T05:50:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.974+0000 7f62870de640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628013db70 con 0x7f6280070230 2024-04-22T05:50:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.974+0000 7f62870de640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6280071270 con 0x7f628006f800 2024-04-22T05:50:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627ffff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6280070230 0x7f628013e2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52638/0 (socket says 172.21.15.62:52638) 2024-04-22T05:50:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627ffff640 1 -- 172.21.15.62:0/2253781684 learned_addr learned my addr 172.21.15.62:0/2253781684 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-22T05:50:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f6284e7a640 1 --2- 172.21.15.62:0/2253781684 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f628006f800 0x7f62800ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261463689 0 0) 0x7f628013db70 con 0x7f6280070230 2024-04-22T05:50:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f626c003530 con 0x7f6280070230 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3555748072 0 0) 0x7f626c003530 con 0x7f6280070230 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f628006f800 msgr2=0x7f62800ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 --2- 172.21.15.62:0/2253781684 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f628006f800 0x7f62800ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 >> v1:172.21.15.40:6789/0 conn(0x7f628013aad0 legacy=0x7f628013af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628013e9f0 con 0x7f6280070230 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 <== mon.1 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268782488 0 0) 0x7f6268001680 con 0x7f6280070230 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 >> v1:172.21.15.62:6789/0 conn(0x7f6280070230 legacy=0x7f628013e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 --2- 172.21.15.62:0/2253781684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f626c0039f0 0x7f626c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 --2- 172.21.15.62:0/2253781684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f626c0047d0 0x7f626c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 --2- 172.21.15.62:0/2253781684 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f628006f800 0x7f626c005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6280071270 con 0x7f626c0047d0 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c00bb50 con 0x7f626c0039f0 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626c00df20 con 0x7f628006f800 2024-04-22T05:50:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f6284e7a640 1 --2- 172.21.15.62:0/2253781684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f626c0039f0 0x7f626c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627ffff640 1 --2- 172.21.15.62:0/2253781684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f626c0047d0 0x7f626c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f628567b640 1 --2- 172.21.15.62:0/2253781684 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f628006f800 0x7f626c005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f6284e7a640 1 -- 172.21.15.62:0/2253781684 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f628006f800 msgr2=0x7f626c005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f6284e7a640 1 --2- 172.21.15.62:0/2253781684 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f628006f800 0x7f626c005240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f6284e7a640 1 -- 172.21.15.62:0/2253781684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f626c0047d0 msgr2=0x7f626c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f6284e7a640 1 --2- 172.21.15.62:0/2253781684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f626c0047d0 0x7f626c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f6284e7a640 1 -- 172.21.15.62:0/2253781684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c003530 con 0x7f626c0039f0 2024-04-22T05:50:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627ffff640 1 --2- 172.21.15.62:0/2253781684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f626c0047d0 0x7f626c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:50:28.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f6284e7a640 1 --2- 172.21.15.62:0/2253781684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f626c0039f0 0x7f626c003e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6278009ae0 tx=0x7f627801ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f918187c55829b22 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:28.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780a6410 con 0x7f626c0039f0 2024-04-22T05:50:28.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62780a6570 con 0x7f626c0039f0 2024-04-22T05:50:28.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.978+0000 7f627f7fe640 1 -- 172.21.15.62:0/2253781684 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780a6920 con 0x7f626c0039f0 2024-04-22T05:50:28.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 -- 172.21.15.62:0/2253781684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f626c0039f0 msgr2=0x7f626c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 --2- 172.21.15.62:0/2253781684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f626c0039f0 0x7f626c003e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6278009ae0 tx=0x7f627801ca20 comp rx=0 tx=0).stop 2024-04-22T05:50:28.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f628567b640 1 -- 172.21.15.62:0/2253781684 reap_dead start 2024-04-22T05:50:28.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 -- 172.21.15.62:0/2253781684 shutdown_connections 2024-04-22T05:50:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 -- 172.21.15.62:0/2253781684 >> 172.21.15.62:0/2253781684 conn(0x7f62800b3920 msgr2=0x7f62800b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 -- 172.21.15.62:0/2253781684 shutdown_connections 2024-04-22T05:50:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 -- 172.21.15.62:0/2253781684 wait complete. 2024-04-22T05:50:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 Processor -- start 2024-04-22T05:50:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 -- start start 2024-04-22T05:50:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f626c0039f0 0x7f628013d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62800a7450 0x7f62801d54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62800acbe0 0x7f62801d9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62801d8ce0 con 0x7f62800acbe0 2024-04-22T05:50:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62801d89b0 con 0x7f62800a7450 2024-04-22T05:50:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62801d96b0 con 0x7f626c0039f0 2024-04-22T05:50:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62800a7450 0x7f62801d54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62800a7450 0x7f62801d54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38386/0 (socket says 172.21.15.62:38386) 2024-04-22T05:50:28.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627ffff640 1 -- 172.21.15.62:0/1405285255 learned_addr learned my addr 172.21.15.62:0/1405285255 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-22T05:50:28.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f6284e7a640 1 --2- 172.21.15.62:0/1405285255 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f626c0039f0 0x7f628013d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f628567b640 1 --2- 172.21.15.62:0/1405285255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62800acbe0 0x7f62801d9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:28.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627ffff640 1 -- 172.21.15.62:0/1405285255 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f626c0039f0 msgr2=0x7f628013d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627ffff640 1 --2- 172.21.15.62:0/1405285255 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f626c0039f0 0x7f628013d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627ffff640 1 -- 172.21.15.62:0/1405285255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62800acbe0 msgr2=0x7f62801d9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:28.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627ffff640 1 --2- 172.21.15.62:0/1405285255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62800acbe0 0x7f62801d9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:28.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627ffff640 1 -- 172.21.15.62:0/1405285255 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62801d9870 con 0x7f62800a7450 2024-04-22T05:50:28.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f628567b640 1 --2- 172.21.15.62:0/1405285255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62800acbe0 0x7f62801d9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:50:28.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f6284e7a640 1 --2- 172.21.15.62:0/1405285255 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f626c0039f0 0x7f628013d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T05:50:28.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627ffff640 1 --2- 172.21.15.62:0/1405285255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62800a7450 0x7f62801d54b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f626800d830 tx=0x7f626800dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:28.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627d7fa640 1 -- 172.21.15.62:0/1405285255 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62680074e0 con 0x7f62800a7450 2024-04-22T05:50:28.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627d7fa640 1 -- 172.21.15.62:0/1405285255 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6268007640 con 0x7f62800a7450 2024-04-22T05:50:28.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627d7fa640 1 -- 172.21.15.62:0/1405285255 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268004a10 con 0x7f62800a7450 2024-04-22T05:50:28.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 -- 172.21.15.62:0/1405285255 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62800abed0 con 0x7f62800a7450 2024-04-22T05:50:28.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 -- 172.21.15.62:0/1405285255 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62800ac380 con 0x7f62800a7450 2024-04-22T05:50:28.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627d7fa640 1 -- 172.21.15.62:0/1405285255 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f6268004b70 con 0x7f62800a7450 2024-04-22T05:50:28.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f627d7fa640 1 -- 172.21.15.62:0/1405285255 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f6268013820 con 0x7f62800a7450 2024-04-22T05:50:28.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.982+0000 7f62870de640 1 -- 172.21.15.62:0/1405285255 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6280066d70 con 0x7f62800a7450 2024-04-22T05:50:28.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:28.986+0000 7f627d7fa640 1 -- 172.21.15.62:0/1405285255 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f6268022060 con 0x7f62800a7450 2024-04-22T05:50:29.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.054+0000 7f62870de640 1 -- 172.21.15.62:0/1405285255 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cb6b2995-f7ce-4a68-ac16-d1690cc2b9ac", "id": 1} v 0) v1 -- 0x7f62800b2c30 con 0x7f62800a7450 2024-04-22T05:50:29.057 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:29.058+0000 7f509a89e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T05:50:29.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.058+0000 7f627d7fa640 1 -- 172.21.15.62:0/1405285255 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cb6b2995-f7ce-4a68-ac16-d1690cc2b9ac", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f626802b0b0 con 0x7f62800a7450 2024-04-22T05:50:29.060 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-22T05:50:29.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.062+0000 7f62870de640 1 -- 172.21.15.62:0/1405285255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62800a7450 msgr2=0x7f62801d54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.062+0000 7f62870de640 1 --2- 172.21.15.62:0/1405285255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62800a7450 0x7f62801d54b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f626800d830 tx=0x7f626800dcf0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.062+0000 7f62870de640 1 -- 172.21.15.62:0/1405285255 shutdown_connections 2024-04-22T05:50:29.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.062+0000 7f62870de640 1 --2- 172.21.15.62:0/1405285255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62800a7450 0x7f62801d54b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.062+0000 7f62870de640 1 --2- 172.21.15.62:0/1405285255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62800acbe0 0x7f62801d9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.062+0000 7f62870de640 1 --2- 172.21.15.62:0/1405285255 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f626c0039f0 0x7f628013d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.062+0000 7f62870de640 1 -- 172.21.15.62:0/1405285255 >> 172.21.15.62:0/1405285255 conn(0x7f62800b3920 msgr2=0x7f6280065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:29.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.062+0000 7f62870de640 1 -- 172.21.15.62:0/1405285255 shutdown_connections 2024-04-22T05:50:29.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.062+0000 7f62870de640 1 -- 172.21.15.62:0/1405285255 wait complete. 2024-04-22T05:50:29.077 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new a5634ce0-a08d-403f-bb05-eadb21417851 2 2024-04-22T05:50:29.153 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:29.150+0000 7f7ca365d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T05:50:29.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc874c3640 1 Processor -- start 2024-04-22T05:50:29.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc874c3640 1 -- start start 2024-04-22T05:50:29.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc874c3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc800703a0 0x7fdc8006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc874c3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc800ae750 0x7fdc80144770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc874c3640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc80071300 0x7fdc80144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc874c3640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc80070c80 con 0x7fdc800703a0 2024-04-22T05:50:29.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc874c3640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8006b930 con 0x7fdc80071300 2024-04-22T05:50:29.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc874c3640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc80071d30 con 0x7fdc800ae750 2024-04-22T05:50:29.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc85a60640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc80071300 0x7fdc80144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc85a60640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc80071300 0x7fdc80144cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38402/0 (socket says 172.21.15.62:38402) 2024-04-22T05:50:29.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc85a60640 1 -- 172.21.15.62:0/2885577643 learned_addr learned my addr 172.21.15.62:0/2885577643 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-22T05:50:29.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc84a5e640 1 --2- 172.21.15.62:0/2885577643 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc800ae750 0x7fdc80144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc8525f640 1 --2- 172.21.15.62:0/2885577643 >> v2:172.21.15.40:3300/0 conn(0x7fdc800703a0 0x7fdc8006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc85a60640 1 -- 172.21.15.62:0/2885577643 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc800ae750 msgr2=0x7fdc80144770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:29.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc85a60640 1 --2- 172.21.15.62:0/2885577643 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc800ae750 0x7fdc80144770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc85a60640 1 -- 172.21.15.62:0/2885577643 >> v2:172.21.15.40:3300/0 conn(0x7fdc800703a0 msgr2=0x7fdc8006bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:29.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc85a60640 1 --2- 172.21.15.62:0/2885577643 >> v2:172.21.15.40:3300/0 conn(0x7fdc800703a0 0x7fdc8006bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.190+0000 7fdc85a60640 1 -- 172.21.15.62:0/2885577643 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc80071ea0 con 0x7fdc80071300 2024-04-22T05:50:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc85a60640 1 --2- 172.21.15.62:0/2885577643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc80071300 0x7fdc80144cb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdc8006cc60 tx=0x7fdc7801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1db26cb96336ba3c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc77fff640 1 -- 172.21.15.62:0/2885577643 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780a57e0 con 0x7fdc80071300 2024-04-22T05:50:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc77fff640 1 -- 172.21.15.62:0/2885577643 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc780a5940 con 0x7fdc80071300 2024-04-22T05:50:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc77fff640 1 -- 172.21.15.62:0/2885577643 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780a5cd0 con 0x7fdc80071300 2024-04-22T05:50:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc874c3640 1 -- 172.21.15.62:0/2885577643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc80071300 msgr2=0x7fdc80144cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc874c3640 1 --2- 172.21.15.62:0/2885577643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc80071300 0x7fdc80144cb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdc8006cc60 tx=0x7fdc7801c940 comp rx=0 tx=0).stop 2024-04-22T05:50:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc874c3640 1 -- 172.21.15.62:0/2885577643 shutdown_connections 2024-04-22T05:50:29.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc874c3640 1 --2- 172.21.15.62:0/2885577643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc80071300 0x7fdc80144cb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc874c3640 1 --2- 172.21.15.62:0/2885577643 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc800ae750 0x7fdc80144770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc874c3640 1 --2- 172.21.15.62:0/2885577643 >> v2:172.21.15.40:3300/0 conn(0x7fdc800703a0 0x7fdc8006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc874c3640 1 -- 172.21.15.62:0/2885577643 >> 172.21.15.62:0/2885577643 conn(0x7fdc800b4a10 msgr2=0x7fdc800b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:29.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc874c3640 1 -- 172.21.15.62:0/2885577643 shutdown_connections 2024-04-22T05:50:29.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc874c3640 1 -- 172.21.15.62:0/2885577643 wait complete. 2024-04-22T05:50:29.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc874c3640 1 Processor -- start 2024-04-22T05:50:29.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc874c3640 1 -- start start 2024-04-22T05:50:29.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.194+0000 7fdc874c3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc800703a0 0x7fdc80140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc874c3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc80071300 0x7fdc8013a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc874c3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc800ae750 0x7fdc8013a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc874c3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8013e870 con 0x7fdc800ae750 2024-04-22T05:50:29.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc874c3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc801412b0 con 0x7fdc800703a0 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc874c3640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8013d660 con 0x7fdc80071300 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc8525f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc800703a0 0x7fdc80140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc8525f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc800703a0 0x7fdc80140bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38408/0 (socket says 172.21.15.62:38408) 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc8525f640 1 -- 172.21.15.62:0/2938965194 learned_addr learned my addr 172.21.15.62:0/2938965194 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc85a60640 1 --2- 172.21.15.62:0/2938965194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc800ae750 0x7fdc8013a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc84a5e640 1 --2- 172.21.15.62:0/2938965194 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc80071300 0x7fdc8013a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc8525f640 1 -- 172.21.15.62:0/2938965194 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc80071300 msgr2=0x7fdc8013a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc8525f640 1 --2- 172.21.15.62:0/2938965194 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc80071300 0x7fdc8013a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc8525f640 1 -- 172.21.15.62:0/2938965194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc800ae750 msgr2=0x7fdc8013a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc8525f640 1 --2- 172.21.15.62:0/2938965194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc800ae750 0x7fdc8013a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc8525f640 1 -- 172.21.15.62:0/2938965194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78009b30 con 0x7fdc800703a0 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc84a5e640 1 --2- 172.21.15.62:0/2938965194 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc80071300 0x7fdc8013a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc8525f640 1 --2- 172.21.15.62:0/2938965194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc800703a0 0x7fdc80140bd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00d800 tx=0x7fdc7c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:29.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc75ffb640 1 -- 172.21.15.62:0/2938965194 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c00b6c0 con 0x7fdc800703a0 2024-04-22T05:50:29.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc75ffb640 1 -- 172.21.15.62:0/2938965194 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c00b820 con 0x7fdc800703a0 2024-04-22T05:50:29.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc75ffb640 1 -- 172.21.15.62:0/2938965194 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c00bb90 con 0x7fdc800703a0 2024-04-22T05:50:29.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc874c3640 1 -- 172.21.15.62:0/2938965194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8013da40 con 0x7fdc800703a0 2024-04-22T05:50:29.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc874c3640 1 -- 172.21.15.62:0/2938965194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc801dd000 con 0x7fdc800703a0 2024-04-22T05:50:29.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc75ffb640 1 -- 172.21.15.62:0/2938965194 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fdc7c01e460 con 0x7fdc800703a0 2024-04-22T05:50:29.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc75ffb640 1 -- 172.21.15.62:0/2938965194 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fdc7c014990 con 0x7fdc800703a0 2024-04-22T05:50:29.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.198+0000 7fdc874c3640 1 -- 172.21.15.62:0/2938965194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8013d880 con 0x7fdc800703a0 2024-04-22T05:50:29.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.202+0000 7fdc75ffb640 1 -- 172.21.15.62:0/2938965194 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fdc8013d880 con 0x7fdc800703a0 2024-04-22T05:50:29.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.266+0000 7fdc874c3640 1 -- 172.21.15.62:0/2938965194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a5634ce0-a08d-403f-bb05-eadb21417851", "id": 2} v 0) v1 -- 0x7fdc80066d70 con 0x7fdc800703a0 2024-04-22T05:50:29.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.274+0000 7fdc75ffb640 1 -- 172.21.15.62:0/2938965194 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a5634ce0-a08d-403f-bb05-eadb21417851", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdc7c0147b0 con 0x7fdc800703a0 2024-04-22T05:50:29.275 INFO:teuthology.orchestra.run.smithi062.stdout:2 2024-04-22T05:50:29.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.274+0000 7fdc874c3640 1 -- 172.21.15.62:0/2938965194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc800703a0 msgr2=0x7fdc80140bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.274+0000 7fdc874c3640 1 --2- 172.21.15.62:0/2938965194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc800703a0 0x7fdc80140bd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00d800 tx=0x7fdc7c00dcc0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.274+0000 7fdc874c3640 1 -- 172.21.15.62:0/2938965194 shutdown_connections 2024-04-22T05:50:29.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.274+0000 7fdc874c3640 1 --2- 172.21.15.62:0/2938965194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc800ae750 0x7fdc8013a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.274+0000 7fdc874c3640 1 --2- 172.21.15.62:0/2938965194 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc80071300 0x7fdc8013a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.274+0000 7fdc874c3640 1 --2- 172.21.15.62:0/2938965194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc800703a0 0x7fdc80140bd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.274+0000 7fdc874c3640 1 -- 172.21.15.62:0/2938965194 >> 172.21.15.62:0/2938965194 conn(0x7fdc800b4a10 msgr2=0x7fdc800afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:29.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.274+0000 7fdc874c3640 1 -- 172.21.15.62:0/2938965194 shutdown_connections 2024-04-22T05:50:29.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.274+0000 7fdc874c3640 1 -- 172.21.15.62:0/2938965194 wait complete. 2024-04-22T05:50:29.291 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new bada948d-c18e-4bf2-8902-9f383fcd1b26 3 2024-04-22T05:50:29.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe076b0c640 1 Processor -- start 2024-04-22T05:50:29.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe076b0c640 1 -- start start 2024-04-22T05:50:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe076b0c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0700aed30 0x7fe07013be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe076b0c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe0700afa10 0x7fe07013c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe076b0c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe070071660 con 0x7fe070071280 2024-04-22T05:50:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe076b0c640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0700af6a0 con 0x7fe0700afa10 2024-04-22T05:50:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe076b0c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700ad510 con 0x7fe0700aed30 2024-04-22T05:50:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe0750a9640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe0700afa10 0x7fe07013c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe0750a9640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe0700afa10 0x7fe07013c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38422/0 (socket says 172.21.15.62:38422) 2024-04-22T05:50:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe0750a9640 1 -- 172.21.15.62:0/2332611020 learned_addr learned my addr 172.21.15.62:0/2332611020 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-22T05:50:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe06ffff640 1 --2- 172.21.15.62:0/2332611020 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0700aed30 0x7fe07013be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe0750a9640 1 -- 172.21.15.62:0/2332611020 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0700aed30 msgr2=0x7fe07013be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:29.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe0750a9640 1 --2- 172.21.15.62:0/2332611020 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0700aed30 0x7fe07013be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe0750a9640 1 -- 172.21.15.62:0/2332611020 >> v1:172.21.15.40:6789/0 conn(0x7fe070071280 legacy=0x7fe0700ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe0750a9640 1 -- 172.21.15.62:0/2332611020 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07013c900 con 0x7fe0700afa10 2024-04-22T05:50:29.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe0750a9640 1 --2- 172.21.15.62:0/2332611020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0700afa10 0x7fe07013c3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe068009e50 tx=0x7fe06801c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f0d49f566c80b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:29.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe06f7fe640 1 -- 172.21.15.62:0/2332611020 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0680a57d0 con 0x7fe0700afa10 2024-04-22T05:50:29.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe06f7fe640 1 -- 172.21.15.62:0/2332611020 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0680a5930 con 0x7fe0700afa10 2024-04-22T05:50:29.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.398+0000 7fe06f7fe640 1 -- 172.21.15.62:0/2332611020 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0680a5cc0 con 0x7fe0700afa10 2024-04-22T05:50:29.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 -- 172.21.15.62:0/2332611020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0700afa10 msgr2=0x7fe07013c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 --2- 172.21.15.62:0/2332611020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0700afa10 0x7fe07013c3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe068009e50 tx=0x7fe06801c8f0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 -- 172.21.15.62:0/2332611020 shutdown_connections 2024-04-22T05:50:29.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 --2- 172.21.15.62:0/2332611020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0700afa10 0x7fe07013c3c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 --2- 172.21.15.62:0/2332611020 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0700aed30 0x7fe07013be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 -- 172.21.15.62:0/2332611020 >> 172.21.15.62:0/2332611020 conn(0x7fe0700b3920 msgr2=0x7fe0700b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:29.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 -- 172.21.15.62:0/2332611020 shutdown_connections 2024-04-22T05:50:29.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 -- 172.21.15.62:0/2332611020 wait complete. 2024-04-22T05:50:29.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 Processor -- start 2024-04-22T05:50:29.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 -- start start 2024-04-22T05:50:29.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe070071280 0x7fe070134bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0700aed30 0x7fe0701da170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0700afa10 0x7fe070130b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe070135540 con 0x7fe070071280 2024-04-22T05:50:29.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0701daa40 con 0x7fe0700afa10 2024-04-22T05:50:29.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0701da6b0 con 0x7fe0700aed30 2024-04-22T05:50:29.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe0750a9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0700afa10 0x7fe070130b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe0750a9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0700afa10 0x7fe070130b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38424/0 (socket says 172.21.15.62:38424) 2024-04-22T05:50:29.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe0750a9640 1 -- 172.21.15.62:0/2267676439 learned_addr learned my addr 172.21.15.62:0/2267676439 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-22T05:50:29.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe06ffff640 1 --2- 172.21.15.62:0/2267676439 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0700aed30 0x7fe0701da170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe0748a8640 1 --2- 172.21.15.62:0/2267676439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe070071280 0x7fe070134bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe0750a9640 1 -- 172.21.15.62:0/2267676439 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0700aed30 msgr2=0x7fe0701da170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe0750a9640 1 --2- 172.21.15.62:0/2267676439 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0700aed30 0x7fe0701da170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe0750a9640 1 -- 172.21.15.62:0/2267676439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe070071280 msgr2=0x7fe070134bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe0750a9640 1 --2- 172.21.15.62:0/2267676439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe070071280 0x7fe070134bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe0750a9640 1 -- 172.21.15.62:0/2267676439 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe068009b30 con 0x7fe0700afa10 2024-04-22T05:50:29.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe0750a9640 1 --2- 172.21.15.62:0/2267676439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0700afa10 0x7fe070130b10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe06801a520 tx=0x7fe068019690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:29.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe06d7fa640 1 -- 172.21.15.62:0/2267676439 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0680b8640 con 0x7fe0700afa10 2024-04-22T05:50:29.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe06d7fa640 1 -- 172.21.15.62:0/2267676439 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0680b87a0 con 0x7fe0700afa10 2024-04-22T05:50:29.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 -- 172.21.15.62:0/2267676439 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07013df00 con 0x7fe0700afa10 2024-04-22T05:50:29.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe06d7fa640 1 -- 172.21.15.62:0/2267676439 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0680b8b50 con 0x7fe0700afa10 2024-04-22T05:50:29.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe076b0c640 1 -- 172.21.15.62:0/2267676439 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe070131220 con 0x7fe0700afa10 2024-04-22T05:50:29.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe06d7fa640 1 -- 172.21.15.62:0/2267676439 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fe0680c1450 con 0x7fe0700afa10 2024-04-22T05:50:29.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.402+0000 7fe06d7fa640 1 -- 172.21.15.62:0/2267676439 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fe0680b68a0 con 0x7fe0700afa10 2024-04-22T05:50:29.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.406+0000 7fe076b0c640 1 -- 172.21.15.62:0/2267676439 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe070066d70 con 0x7fe0700afa10 2024-04-22T05:50:29.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.406+0000 7fe06d7fa640 1 -- 172.21.15.62:0/2267676439 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fe0680e0dc0 con 0x7fe0700afa10 2024-04-22T05:50:29.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.474+0000 7fe076b0c640 1 -- 172.21.15.62:0/2267676439 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bada948d-c18e-4bf2-8902-9f383fcd1b26", "id": 3} v 0) v1 -- 0x7fe0701dab80 con 0x7fe0700afa10 2024-04-22T05:50:29.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.478+0000 7fe06d7fa640 1 -- 172.21.15.62:0/2267676439 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bada948d-c18e-4bf2-8902-9f383fcd1b26", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe0701dab80 con 0x7fe0700afa10 2024-04-22T05:50:29.480 INFO:teuthology.orchestra.run.smithi062.stdout:3 2024-04-22T05:50:29.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.478+0000 7fe076b0c640 1 -- 172.21.15.62:0/2267676439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0700afa10 msgr2=0x7fe070130b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.478+0000 7fe076b0c640 1 --2- 172.21.15.62:0/2267676439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0700afa10 0x7fe070130b10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe06801a520 tx=0x7fe068019690 comp rx=0 tx=0).stop 2024-04-22T05:50:29.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.478+0000 7fe076b0c640 1 -- 172.21.15.62:0/2267676439 shutdown_connections 2024-04-22T05:50:29.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.478+0000 7fe076b0c640 1 --2- 172.21.15.62:0/2267676439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0700afa10 0x7fe070130b10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.478+0000 7fe076b0c640 1 --2- 172.21.15.62:0/2267676439 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe0700aed30 0x7fe0701da170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.478+0000 7fe076b0c640 1 --2- 172.21.15.62:0/2267676439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe070071280 0x7fe070134bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.482+0000 7fe076b0c640 1 -- 172.21.15.62:0/2267676439 >> 172.21.15.62:0/2267676439 conn(0x7fe0700b3920 msgr2=0x7fe0700b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:29.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.482+0000 7fe076b0c640 1 -- 172.21.15.62:0/2267676439 shutdown_connections 2024-04-22T05:50:29.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.482+0000 7fe076b0c640 1 -- 172.21.15.62:0/2267676439 wait complete. 2024-04-22T05:50:29.496 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 4ad76cfa-9e85-421e-bd2e-61006dd694bd 4 2024-04-22T05:50:29.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.606+0000 7f53e655d640 1 Processor -- start 2024-04-22T05:50:29.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.606+0000 7f53e655d640 1 -- start start 2024-04-22T05:50:29.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.606+0000 7f53e655d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f53e013ac00 0x7f53e013da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.606+0000 7f53e655d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e0071160 con 0x7f53e0071bc0 2024-04-22T05:50:29.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.606+0000 7f53e655d640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e013df50 con 0x7f53e0071fa0 2024-04-22T05:50:29.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.606+0000 7f53e655d640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e006f310 con 0x7f53e013ac00 2024-04-22T05:50:29.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.606+0000 7f53df7fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f53e0071fa0 0x7f53e013d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52644/0 (socket says 172.21.15.62:52644) 2024-04-22T05:50:29.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.606+0000 7f53df7fe640 1 -- 172.21.15.62:0/2637820978 learned_addr learned my addr 172.21.15.62:0/2637820978 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-22T05:50:29.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53e4afa640 1 --2- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f53e013ac00 0x7f53e013da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223936322 0 0) 0x7f53e013df50 con 0x7f53e0071fa0 2024-04-22T05:50:29.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c8003530 con 0x7f53e0071fa0 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1741655226 0 0) 0x7f53c8003530 con 0x7f53e0071fa0 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f53e013ac00 msgr2=0x7f53e013da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 --2- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f53e013ac00 0x7f53e013da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 >> v1:172.21.15.40:6789/0 conn(0x7f53e0071bc0 legacy=0x7f53e013a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e006ff70 con 0x7f53e0071fa0 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 <== mon.1 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268782488 0 0) 0x7f53d4001680 con 0x7f53e0071fa0 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 >> v1:172.21.15.62:6789/0 conn(0x7f53e0071fa0 legacy=0x7f53e013d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 --2- 172.21.15.62:0/2637820978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c80039f0 0x7f53c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 --2- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f53e013ac00 0x7f53c80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 --2- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8004d10 0x7f53c8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e006f310 con 0x7f53c8004d10 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c800dc50 con 0x7f53c80039f0 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c800ddc0 con 0x7f53e013ac00 2024-04-22T05:50:29.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53dffff640 1 --2- 172.21.15.62:0/2637820978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c80039f0 0x7f53c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53df7fe640 1 --2- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f53e013ac00 0x7f53c80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53e4afa640 1 -- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8004d10 msgr2=0x7f53c8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-22T05:50:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53e4afa640 1 -- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8004d10 msgr2=0x7f53c8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53e4afa640 1 --2- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8004d10 0x7f53c8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2024-04-22T05:50:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53e4afa640 1 --2- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8004d10 0x7f53c8005190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T05:50:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53dffff640 1 -- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f53e013ac00 msgr2=0x7f53c80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53dffff640 1 --2- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f53e013ac00 0x7f53c80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53dffff640 1 -- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8004d10 msgr2=0x7f53c8005190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53dffff640 1 --2- 172.21.15.62:0/2637820978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53c8004d10 0x7f53c8005190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.610+0000 7f53dffff640 1 -- 172.21.15.62:0/2637820978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c8003530 con 0x7f53c80039f0 2024-04-22T05:50:29.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53dffff640 1 --2- 172.21.15.62:0/2637820978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c80039f0 0x7f53c8003dd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53d0001910 tx=0x7f53d001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ae097aa92896a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:29.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d00ad640 con 0x7f53c80039f0 2024-04-22T05:50:29.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d00ad7c0 con 0x7f53c80039f0 2024-04-22T05:50:29.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53deffd640 1 -- 172.21.15.62:0/2637820978 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d00adb70 con 0x7f53c80039f0 2024-04-22T05:50:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 -- 172.21.15.62:0/2637820978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c80039f0 msgr2=0x7f53c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 --2- 172.21.15.62:0/2637820978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c80039f0 0x7f53c8003dd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53d0001910 tx=0x7f53d001c560 comp rx=0 tx=0).stop 2024-04-22T05:50:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e4afa640 1 -- 172.21.15.62:0/2637820978 reap_dead start 2024-04-22T05:50:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 -- 172.21.15.62:0/2637820978 shutdown_connections 2024-04-22T05:50:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 -- 172.21.15.62:0/2637820978 >> 172.21.15.62:0/2637820978 conn(0x7f53e00b3270 msgr2=0x7f53e00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 -- 172.21.15.62:0/2637820978 shutdown_connections 2024-04-22T05:50:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 -- 172.21.15.62:0/2637820978 wait complete. 2024-04-22T05:50:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 Processor -- start 2024-04-22T05:50:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 -- start start 2024-04-22T05:50:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f53c80039f0 0x7f53e01c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e013f630 0x7f53e01402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e0140820 0x7f53e01342d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0141300 con 0x7f53e013f630 2024-04-22T05:50:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01352b0 con 0x7f53e0140820 2024-04-22T05:50:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01414e0 con 0x7f53c80039f0 2024-04-22T05:50:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e4afa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e0140820 0x7f53e01342d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e4afa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e0140820 0x7f53e01342d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38440/0 (socket says 172.21.15.62:38440) 2024-04-22T05:50:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e4afa640 1 -- 172.21.15.62:0/3570941275 learned_addr learned my addr 172.21.15.62:0/3570941275 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-22T05:50:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53df7fe640 1 --2- 172.21.15.62:0/3570941275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e013f630 0x7f53e01402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53dffff640 1 --2- 172.21.15.62:0/3570941275 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f53c80039f0 0x7f53e01c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e4afa640 1 -- 172.21.15.62:0/3570941275 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f53c80039f0 msgr2=0x7f53e01c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e4afa640 1 --2- 172.21.15.62:0/3570941275 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f53c80039f0 0x7f53e01c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e4afa640 1 -- 172.21.15.62:0/3570941275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e013f630 msgr2=0x7f53e01402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e4afa640 1 --2- 172.21.15.62:0/3570941275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e013f630 0x7f53e01402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e4afa640 1 -- 172.21.15.62:0/3570941275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e013e820 con 0x7f53e0140820 2024-04-22T05:50:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e4afa640 1 --2- 172.21.15.62:0/3570941275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e0140820 0x7f53e01342d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f53e013afe0 tx=0x7f53d80075c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53dcff9640 1 -- 172.21.15.62:0/3570941275 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8011070 con 0x7f53e0140820 2024-04-22T05:50:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53dcff9640 1 -- 172.21.15.62:0/3570941275 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d8007db0 con 0x7f53e0140820 2024-04-22T05:50:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53dcff9640 1 -- 172.21.15.62:0/3570941275 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d800adf0 con 0x7f53e0140820 2024-04-22T05:50:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 -- 172.21.15.62:0/3570941275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e0134f40 con 0x7f53e0140820 2024-04-22T05:50:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.614+0000 7f53e655d640 1 -- 172.21.15.62:0/3570941275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e013fc80 con 0x7f53e0140820 2024-04-22T05:50:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.618+0000 7f53dcff9640 1 -- 172.21.15.62:0/3570941275 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f53d8016690 con 0x7f53e0140820 2024-04-22T05:50:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.618+0000 7f53dcff9640 1 -- 172.21.15.62:0/3570941275 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f53d8014880 con 0x7f53e0140820 2024-04-22T05:50:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.618+0000 7f53e655d640 1 -- 172.21.15.62:0/3570941275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e0140000 con 0x7f53e0140820 2024-04-22T05:50:29.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.622+0000 7f53dcff9640 1 -- 172.21.15.62:0/3570941275 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f53e0140000 con 0x7f53e0140820 2024-04-22T05:50:29.641 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:29.638+0000 7f509a89e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T05:50:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.690+0000 7f53e655d640 1 -- 172.21.15.62:0/3570941275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4ad76cfa-9e85-421e-bd2e-61006dd694bd", "id": 4} v 0) v1 -- 0x7f53e0066d70 con 0x7f53e0140820 2024-04-22T05:50:29.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.694+0000 7f53dcff9640 1 -- 172.21.15.62:0/3570941275 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4ad76cfa-9e85-421e-bd2e-61006dd694bd", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f53d801b070 con 0x7f53e0140820 2024-04-22T05:50:29.697 INFO:teuthology.orchestra.run.smithi062.stdout:4 2024-04-22T05:50:29.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.698+0000 7f53e655d640 1 -- 172.21.15.62:0/3570941275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e0140820 msgr2=0x7f53e01342d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.698+0000 7f53e655d640 1 --2- 172.21.15.62:0/3570941275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e0140820 0x7f53e01342d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f53e013afe0 tx=0x7f53d80075c0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.698 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:29.698+0000 7f509a89e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T05:50:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.698+0000 7f53e655d640 1 -- 172.21.15.62:0/3570941275 shutdown_connections 2024-04-22T05:50:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.698+0000 7f53e655d640 1 --2- 172.21.15.62:0/3570941275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e0140820 0x7f53e01342d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.698+0000 7f53e655d640 1 --2- 172.21.15.62:0/3570941275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e013f630 0x7f53e01402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.698+0000 7f53e655d640 1 --2- 172.21.15.62:0/3570941275 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f53c80039f0 0x7f53e01c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.698+0000 7f53e655d640 1 -- 172.21.15.62:0/3570941275 >> 172.21.15.62:0/3570941275 conn(0x7f53e00b3270 msgr2=0x7f53e00ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.698+0000 7f53e655d640 1 -- 172.21.15.62:0/3570941275 shutdown_connections 2024-04-22T05:50:29.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.698+0000 7f53e655d640 1 -- 172.21.15.62:0/3570941275 wait complete. 2024-04-22T05:50:29.714 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new b3b6f5a2-d949-429c-bfc4-b6a7333d8184 5 2024-04-22T05:50:29.748 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:29.746+0000 7f7ca365d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T05:50:29.754 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:29.754+0000 7f509a89e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T05:50:29.804 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:29.802+0000 7f7ca365d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T05:50:29.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.826+0000 7f0dfc607640 1 Processor -- start 2024-04-22T05:50:29.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.826+0000 7f0dfc607640 1 -- start start 2024-04-22T05:50:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.826+0000 7f0dfc607640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0df406a0d0 0x7f0df4140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.826+0000 7f0dfc607640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0df413ab70 0x7f0df413afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.826+0000 7f0dfc607640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df406a4b0 con 0x7f0df40afbf0 2024-04-22T05:50:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.826+0000 7f0dfc607640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df40ace30 con 0x7f0df413ab70 2024-04-22T05:50:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.826+0000 7f0dfc607640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df40ad110 con 0x7f0df406a0d0 2024-04-22T05:50:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0df9ba2640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0df40afbf0 0x7f0df40adc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52646/0 (socket says 172.21.15.62:52646) 2024-04-22T05:50:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0df9ba2640 1 -- 172.21.15.62:0/3036582682 learned_addr learned my addr 172.21.15.62:0/3036582682 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-22T05:50:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0dfaba4640 1 --2- 172.21.15.62:0/3036582682 >> v2:172.21.15.62:3300/0 conn(0x7f0df413ab70 0x7f0df413afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0dfa3a3640 1 --2- 172.21.15.62:0/3036582682 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0df406a0d0 0x7f0df4140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0df93a1640 1 -- 172.21.15.62:0/3036582682 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365678333 0 0) 0x7f0df406a4b0 con 0x7f0df40afbf0 2024-04-22T05:50:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0df93a1640 1 -- 172.21.15.62:0/3036582682 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0de0003530 con 0x7f0df40afbf0 2024-04-22T05:50:29.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0dfaba4640 1 -- 172.21.15.62:0/3036582682 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0df406a0d0 msgr2=0x7f0df4140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:29.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0dfaba4640 1 --2- 172.21.15.62:0/3036582682 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0df406a0d0 0x7f0df4140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0dfaba4640 1 -- 172.21.15.62:0/3036582682 >> v1:172.21.15.62:6789/0 conn(0x7f0df40afbf0 legacy=0x7f0df40adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0dfaba4640 1 -- 172.21.15.62:0/3036582682 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df40ad530 con 0x7f0df413ab70 2024-04-22T05:50:29.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0dfa3a3640 1 --2- 172.21.15.62:0/3036582682 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0df406a0d0 0x7f0df4140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T05:50:29.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0dfaba4640 1 --2- 172.21.15.62:0/3036582682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df413ab70 0x7f0df413afa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009d90 tx=0x7f0dec01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbaa7e439de86b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:29.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0df93a1640 1 -- 172.21.15.62:0/3036582682 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0a5a50 con 0x7f0df413ab70 2024-04-22T05:50:29.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0df93a1640 1 -- 172.21.15.62:0/3036582682 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec0a5bb0 con 0x7f0df413ab70 2024-04-22T05:50:29.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.830+0000 7f0df93a1640 1 -- 172.21.15.62:0/3036582682 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0ad440 con 0x7f0df413ab70 2024-04-22T05:50:29.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 -- 172.21.15.62:0/3036582682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df413ab70 msgr2=0x7f0df413afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 --2- 172.21.15.62:0/3036582682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df413ab70 0x7f0df413afa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009d90 tx=0x7f0dec01c970 comp rx=0 tx=0).stop 2024-04-22T05:50:29.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 -- 172.21.15.62:0/3036582682 shutdown_connections 2024-04-22T05:50:29.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 --2- 172.21.15.62:0/3036582682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df413ab70 0x7f0df413afa0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 --2- 172.21.15.62:0/3036582682 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0df406a0d0 0x7f0df4140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 -- 172.21.15.62:0/3036582682 >> 172.21.15.62:0/3036582682 conn(0x7f0df406ba90 msgr2=0x7f0df40b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:29.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 -- 172.21.15.62:0/3036582682 shutdown_connections 2024-04-22T05:50:29.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 -- 172.21.15.62:0/3036582682 wait complete. 2024-04-22T05:50:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 Processor -- start 2024-04-22T05:50:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 -- start start 2024-04-22T05:50:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0df406a0d0 0x7f0df4139000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0df40afbf0 0x7f0df41dc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df413ab70 0x7f0df41dc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df41d96d0 con 0x7f0df406a0d0 2024-04-22T05:50:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df41dcc20 con 0x7f0df413ab70 2024-04-22T05:50:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df41d9b60 con 0x7f0df40afbf0 2024-04-22T05:50:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfaba4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df413ab70 0x7f0df41dc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfaba4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df413ab70 0x7f0df41dc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38450/0 (socket says 172.21.15.62:38450) 2024-04-22T05:50:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfaba4640 1 -- 172.21.15.62:0/2585643817 learned_addr learned my addr 172.21.15.62:0/2585643817 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-22T05:50:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0df9ba2640 1 --2- 172.21.15.62:0/2585643817 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0df40afbf0 0x7f0df41dc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfa3a3640 1 --2- 172.21.15.62:0/2585643817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0df406a0d0 0x7f0df4139000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfaba4640 1 -- 172.21.15.62:0/2585643817 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0df40afbf0 msgr2=0x7f0df41dc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfaba4640 1 --2- 172.21.15.62:0/2585643817 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0df40afbf0 0x7f0df41dc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfaba4640 1 -- 172.21.15.62:0/2585643817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0df406a0d0 msgr2=0x7f0df4139000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfaba4640 1 --2- 172.21.15.62:0/2585643817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0df406a0d0 0x7f0df4139000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfaba4640 1 -- 172.21.15.62:0/2585643817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec009a70 con 0x7f0df413ab70 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfa3a3640 1 --2- 172.21.15.62:0/2585643817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0df406a0d0 0x7f0df4139000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfaba4640 1 --2- 172.21.15.62:0/2585643817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df413ab70 0x7f0df41dc6e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0dec01b020 tx=0x7f0dec019710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0deaffd640 1 -- 172.21.15.62:0/2585643817 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0a5a50 con 0x7f0df413ab70 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0deaffd640 1 -- 172.21.15.62:0/2585643817 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec009450 con 0x7f0df413ab70 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 -- 172.21.15.62:0/2585643817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df4139ab0 con 0x7f0df413ab70 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0deaffd640 1 -- 172.21.15.62:0/2585643817 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0b9670 con 0x7f0df413ab70 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0dfc607640 1 -- 172.21.15.62:0/2585643817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df41d9330 con 0x7f0df413ab70 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0deaffd640 1 -- 172.21.15.62:0/2585643817 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0dec0b9ad0 con 0x7f0df413ab70 2024-04-22T05:50:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.834+0000 7f0deaffd640 1 -- 172.21.15.62:0/2585643817 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f0dec0c2b30 con 0x7f0df413ab70 2024-04-22T05:50:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.838+0000 7f0dfc607640 1 -- 172.21.15.62:0/2585643817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df413c090 con 0x7f0df413ab70 2024-04-22T05:50:29.840 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:29.838+0000 7f509a89e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T05:50:29.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.838+0000 7f0deaffd640 1 -- 172.21.15.62:0/2585643817 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f0dec0bd020 con 0x7f0df413ab70 2024-04-22T05:50:29.859 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:29.858+0000 7f7ca365d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T05:50:29.895 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:29.894+0000 7f509a89e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T05:50:29.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.906+0000 7f0dfc607640 1 -- 172.21.15.62:0/2585643817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b3b6f5a2-d949-429c-bfc4-b6a7333d8184", "id": 5} v 0) v1 -- 0x7f0df41d9880 con 0x7f0df413ab70 2024-04-22T05:50:29.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.914+0000 7f0deaffd640 1 -- 172.21.15.62:0/2585643817 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b3b6f5a2-d949-429c-bfc4-b6a7333d8184", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0df41d9880 con 0x7f0df413ab70 2024-04-22T05:50:29.915 INFO:teuthology.orchestra.run.smithi062.stdout:5 2024-04-22T05:50:29.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.914+0000 7f0dfc607640 1 -- 172.21.15.62:0/2585643817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df413ab70 msgr2=0x7f0df41dc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:29.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.914+0000 7f0dfc607640 1 --2- 172.21.15.62:0/2585643817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df413ab70 0x7f0df41dc6e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0dec01b020 tx=0x7f0dec019710 comp rx=0 tx=0).stop 2024-04-22T05:50:29.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.914+0000 7f0dfc607640 1 -- 172.21.15.62:0/2585643817 shutdown_connections 2024-04-22T05:50:29.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.914+0000 7f0dfc607640 1 --2- 172.21.15.62:0/2585643817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0df413ab70 0x7f0df41dc6e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.914+0000 7f0dfc607640 1 --2- 172.21.15.62:0/2585643817 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0df40afbf0 0x7f0df41dc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.914+0000 7f0dfc607640 1 --2- 172.21.15.62:0/2585643817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0df406a0d0 0x7f0df4139000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:29.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.914+0000 7f0dfc607640 1 -- 172.21.15.62:0/2585643817 >> 172.21.15.62:0/2585643817 conn(0x7f0df406ba90 msgr2=0x7f0df40af400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:29.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.914+0000 7f0dfc607640 1 -- 172.21.15.62:0/2585643817 shutdown_connections 2024-04-22T05:50:29.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:29.914+0000 7f0dfc607640 1 -- 172.21.15.62:0/2585643817 wait complete. 2024-04-22T05:50:29.932 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 00c5caf5-710a-4162-93fa-917a4823ffaf 6 2024-04-22T05:50:29.940 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:29.938+0000 7f7ca365d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T05:50:29.977 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:29.978+0000 7f509a89e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T05:50:29.994 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:29.994+0000 7f7ca365d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T05:50:30.032 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:30.030+0000 7f509a89e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T05:50:30.076 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:30.074+0000 7f7ca365d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T05:50:30.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.094+0000 7fd39168a640 1 Processor -- start 2024-04-22T05:50:30.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.094+0000 7fd39168a640 1 -- start start 2024-04-22T05:50:30.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.094+0000 7fd39168a640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd38c071280 0x7fd38c0ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.094+0000 7fd39168a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd38c13c620 0x7fd38c1495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.094+0000 7fd39168a640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c0ad3e0 con 0x7fd38c0aecc0 2024-04-22T05:50:30.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.094+0000 7fd39168a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c0ad630 con 0x7fd38c13c620 2024-04-22T05:50:30.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.094+0000 7fd39168a640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b0130 con 0x7fd38c071280 2024-04-22T05:50:30.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.094+0000 7fd38a7fc640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd38c0aecc0 0x7fd38c13bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52648/0 (socket says 172.21.15.62:52648) 2024-04-22T05:50:30.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.094+0000 7fd38b7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd38c13c620 0x7fd38c1495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.094+0000 7fd38a7fc640 1 -- 172.21.15.62:0/2555123413 learned_addr learned my addr 172.21.15.62:0/2555123413 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-22T05:50:30.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.094+0000 7fd38affd640 1 --2- 172.21.15.62:0/2555123413 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd38c071280 0x7fd38c0ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.094+0000 7fd389ffb640 1 -- 172.21.15.62:0/2555123413 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057451762 0 0) 0x7fd38c0ad3e0 con 0x7fd38c0aecc0 2024-04-22T05:50:30.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.094+0000 7fd389ffb640 1 -- 172.21.15.62:0/2555123413 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd378003530 con 0x7fd38c0aecc0 2024-04-22T05:50:30.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.098+0000 7fd38b7fe640 1 -- 172.21.15.62:0/2555123413 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd38c071280 msgr2=0x7fd38c0ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:30.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.098+0000 7fd38b7fe640 1 --2- 172.21.15.62:0/2555123413 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd38c071280 0x7fd38c0ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.098+0000 7fd38b7fe640 1 -- 172.21.15.62:0/2555123413 >> v1:172.21.15.62:6789/0 conn(0x7fd38c0aecc0 legacy=0x7fd38c13bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.098+0000 7fd38b7fe640 1 -- 172.21.15.62:0/2555123413 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0ade80 con 0x7fd38c13c620 2024-04-22T05:50:30.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.098+0000 7fd38b7fe640 1 --2- 172.21.15.62:0/2555123413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd38c13c620 0x7fd38c1495a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0097c0 tx=0x7fd37c01c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=438c54b5d5eaa8d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:30.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.098+0000 7fd389ffb640 1 -- 172.21.15.62:0/2555123413 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0ad680 con 0x7fd38c13c620 2024-04-22T05:50:30.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.098+0000 7fd389ffb640 1 -- 172.21.15.62:0/2555123413 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c0ad7e0 con 0x7fd38c13c620 2024-04-22T05:50:30.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.098+0000 7fd389ffb640 1 -- 172.21.15.62:0/2555123413 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0adb70 con 0x7fd38c13c620 2024-04-22T05:50:30.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 -- 172.21.15.62:0/2555123413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd38c13c620 msgr2=0x7fd38c1495a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 --2- 172.21.15.62:0/2555123413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd38c13c620 0x7fd38c1495a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0097c0 tx=0x7fd37c01c360 comp rx=0 tx=0).stop 2024-04-22T05:50:30.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 -- 172.21.15.62:0/2555123413 shutdown_connections 2024-04-22T05:50:30.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 --2- 172.21.15.62:0/2555123413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd38c13c620 0x7fd38c1495a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 --2- 172.21.15.62:0/2555123413 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd38c071280 0x7fd38c0ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 -- 172.21.15.62:0/2555123413 >> 172.21.15.62:0/2555123413 conn(0x7fd38c0b3920 msgr2=0x7fd38c0b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:30.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 -- 172.21.15.62:0/2555123413 shutdown_connections 2024-04-22T05:50:30.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 -- 172.21.15.62:0/2555123413 wait complete. 2024-04-22T05:50:30.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 Processor -- start 2024-04-22T05:50:30.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 -- start start 2024-04-22T05:50:30.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd38c071280 0x7fd38c13da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd38c0aecc0 0x7fd38c1314d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd38c13c620 0x7fd38c131a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c131f50 con 0x7fd38c0aecc0 2024-04-22T05:50:30.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c13cb90 con 0x7fd38c13c620 2024-04-22T05:50:30.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1323d0 con 0x7fd38c071280 2024-04-22T05:50:30.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd38b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd38c13c620 0x7fd38c131a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd38b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd38c13c620 0x7fd38c131a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38464/0 (socket says 172.21.15.62:38464) 2024-04-22T05:50:30.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd38b7fe640 1 -- 172.21.15.62:0/1515085506 learned_addr learned my addr 172.21.15.62:0/1515085506 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-22T05:50:30.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd38affd640 1 --2- 172.21.15.62:0/1515085506 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd38c071280 0x7fd38c13da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd38a7fc640 1 --2- 172.21.15.62:0/1515085506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd38c0aecc0 0x7fd38c1314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd38b7fe640 1 -- 172.21.15.62:0/1515085506 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd38c071280 msgr2=0x7fd38c13da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd38b7fe640 1 --2- 172.21.15.62:0/1515085506 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd38c071280 0x7fd38c13da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd38b7fe640 1 -- 172.21.15.62:0/1515085506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd38c0aecc0 msgr2=0x7fd38c1314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd38b7fe640 1 --2- 172.21.15.62:0/1515085506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd38c0aecc0 0x7fd38c1314d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd38b7fe640 1 -- 172.21.15.62:0/1515085506 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c009420 con 0x7fd38c13c620 2024-04-22T05:50:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd38a7fc640 1 --2- 172.21.15.62:0/1515085506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd38c0aecc0 0x7fd38c1314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:50:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd38b7fe640 1 --2- 172.21.15.62:0/1515085506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd38c13c620 0x7fd38c131a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0097c0 tx=0x7fd37c019100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd3837fe640 1 -- 172.21.15.62:0/1515085506 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0add60 con 0x7fd38c13c620 2024-04-22T05:50:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd3837fe640 1 -- 172.21.15.62:0/1515085506 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c0b6400 con 0x7fd38c13c620 2024-04-22T05:50:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 -- 172.21.15.62:0/1515085506 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c13d070 con 0x7fd38c13c620 2024-04-22T05:50:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd3837fe640 1 -- 172.21.15.62:0/1515085506 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0b67b0 con 0x7fd38c13c620 2024-04-22T05:50:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 -- 172.21.15.62:0/1515085506 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c070680 con 0x7fd38c13c620 2024-04-22T05:50:30.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd3837fe640 1 -- 172.21.15.62:0/1515085506 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fd37c0b4040 con 0x7fd38c13c620 2024-04-22T05:50:30.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd3837fe640 1 -- 172.21.15.62:0/1515085506 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fd37c0c1d10 con 0x7fd38c13c620 2024-04-22T05:50:30.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.102+0000 7fd39168a640 1 -- 172.21.15.62:0/1515085506 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c1320f0 con 0x7fd38c13c620 2024-04-22T05:50:30.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.106+0000 7fd3837fe640 1 -- 172.21.15.62:0/1515085506 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fd37c0dfdc0 con 0x7fd38c13c620 2024-04-22T05:50:30.131 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:30.130+0000 7f7ca365d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T05:50:30.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.174+0000 7fd39168a640 1 -- 172.21.15.62:0/1515085506 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00c5caf5-710a-4162-93fa-917a4823ffaf", "id": 6} v 0) v1 -- 0x7fd38c066d70 con 0x7fd38c13c620 2024-04-22T05:50:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.182+0000 7fd3837fe640 1 -- 172.21.15.62:0/1515085506 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00c5caf5-710a-4162-93fa-917a4823ffaf", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd37c0ce030 con 0x7fd38c13c620 2024-04-22T05:50:30.182 INFO:teuthology.orchestra.run.smithi062.stdout:6 2024-04-22T05:50:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.182+0000 7fd39168a640 1 -- 172.21.15.62:0/1515085506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd38c13c620 msgr2=0x7fd38c131a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.182+0000 7fd39168a640 1 --2- 172.21.15.62:0/1515085506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd38c13c620 0x7fd38c131a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0097c0 tx=0x7fd37c019100 comp rx=0 tx=0).stop 2024-04-22T05:50:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.182+0000 7fd39168a640 1 -- 172.21.15.62:0/1515085506 shutdown_connections 2024-04-22T05:50:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.182+0000 7fd39168a640 1 --2- 172.21.15.62:0/1515085506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd38c13c620 0x7fd38c131a10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.182+0000 7fd39168a640 1 --2- 172.21.15.62:0/1515085506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd38c0aecc0 0x7fd38c1314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.182+0000 7fd39168a640 1 --2- 172.21.15.62:0/1515085506 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd38c071280 0x7fd38c13da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.182+0000 7fd39168a640 1 -- 172.21.15.62:0/1515085506 >> 172.21.15.62:0/1515085506 conn(0x7fd38c0b3920 msgr2=0x7fd38c139aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.182+0000 7fd39168a640 1 -- 172.21.15.62:0/1515085506 shutdown_connections 2024-04-22T05:50:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.182+0000 7fd39168a640 1 -- 172.21.15.62:0/1515085506 wait complete. 2024-04-22T05:50:30.195 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:30.194+0000 7f509a89e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T05:50:30.198 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new d17323e1-9cec-4bcb-b603-7f9fda35b5f3 7 2024-04-22T05:50:30.258 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:30.258+0000 7f509a89e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T05:50:30.290 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:30.286+0000 7f7ca365d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T05:50:30.351 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:30.350+0000 7f7ca365d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T05:50:30.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.350+0000 7fe692856640 1 Processor -- start 2024-04-22T05:50:30.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.350+0000 7fe692856640 1 -- start start 2024-04-22T05:50:30.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe692856640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe68c13c620 0x7fe68c14a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe692856640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68c071660 con 0x7fe68c071280 2024-04-22T05:50:30.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe692856640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68c0ad3e0 con 0x7fe68c0afa10 2024-04-22T05:50:30.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe692856640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c0ad630 con 0x7fe68c13c620 2024-04-22T05:50:30.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68b7fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe68c0afa10 0x7fe68c13bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52652/0 (socket says 172.21.15.62:52652) 2024-04-22T05:50:30.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68b7fe640 1 -- 172.21.15.62:0/2895737065 learned_addr learned my addr 172.21.15.62:0/2895737065 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-22T05:50:30.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe690df3640 1 --2- 172.21.15.62:0/2895737065 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe68c13c620 0x7fe68c14a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780462744 0 0) 0x7fe68c0ad3e0 con 0x7fe68c0afa10 2024-04-22T05:50:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe674003530 con 0x7fe68c0afa10 2024-04-22T05:50:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1984593321 0 0) 0x7fe674003530 con 0x7fe68c0afa10 2024-04-22T05:50:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe68c13c620 msgr2=0x7fe68c14a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 --2- 172.21.15.62:0/2895737065 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe68c13c620 0x7fe68c14a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 >> v1:172.21.15.40:6789/0 conn(0x7fe68c071280 legacy=0x7fe68c0ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c0b0130 con 0x7fe68c0afa10 2024-04-22T05:50:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 <== mon.1 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268782488 0 0) 0x7fe680001680 con 0x7fe68c0afa10 2024-04-22T05:50:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 >> v1:172.21.15.62:6789/0 conn(0x7fe68c0afa10 legacy=0x7fe68c13bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 --2- 172.21.15.62:0/2895737065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6740039f0 0x7fe674003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 --2- 172.21.15.62:0/2895737065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6740047d0 0x7fe674004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 --2- 172.21.15.62:0/2895737065 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe68c13c620 0x7fe674005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c0ad630 con 0x7fe6740039f0 2024-04-22T05:50:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67400bb20 con 0x7fe6740047d0 2024-04-22T05:50:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67400bcb0 con 0x7fe68c13c620 2024-04-22T05:50:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68b7fe640 1 --2- 172.21.15.62:0/2895737065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6740047d0 0x7fe674004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68bfff640 1 --2- 172.21.15.62:0/2895737065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6740039f0 0x7fe674003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68b7fe640 1 -- 172.21.15.62:0/2895737065 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe68c13c620 msgr2=0x7fe674005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68b7fe640 1 --2- 172.21.15.62:0/2895737065 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe68c13c620 0x7fe674005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68b7fe640 1 -- 172.21.15.62:0/2895737065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6740039f0 msgr2=0x7fe674003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68b7fe640 1 --2- 172.21.15.62:0/2895737065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6740039f0 0x7fe674003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68b7fe640 1 -- 172.21.15.62:0/2895737065 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe674003530 con 0x7fe6740047d0 2024-04-22T05:50:30.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68b7fe640 1 --2- 172.21.15.62:0/2895737065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6740047d0 0x7fe674004c50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe680001060 tx=0x7fe68001cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a09027ca2e318ae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:30.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6800a5900 con 0x7fe6740047d0 2024-04-22T05:50:30.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6800a5a60 con 0x7fe6740047d0 2024-04-22T05:50:30.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.354+0000 7fe68affd640 1 -- 172.21.15.62:0/2895737065 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6800a5e50 con 0x7fe6740047d0 2024-04-22T05:50:30.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 -- 172.21.15.62:0/2895737065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6740047d0 msgr2=0x7fe674004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 --2- 172.21.15.62:0/2895737065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6740047d0 0x7fe674004c50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe680001060 tx=0x7fe68001cc00 comp rx=0 tx=0).stop 2024-04-22T05:50:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe690df3640 1 -- 172.21.15.62:0/2895737065 reap_dead start 2024-04-22T05:50:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 -- 172.21.15.62:0/2895737065 shutdown_connections 2024-04-22T05:50:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 -- 172.21.15.62:0/2895737065 >> 172.21.15.62:0/2895737065 conn(0x7fe68c0b3920 msgr2=0x7fe68c0b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 -- 172.21.15.62:0/2895737065 shutdown_connections 2024-04-22T05:50:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 -- 172.21.15.62:0/2895737065 wait complete. 2024-04-22T05:50:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 Processor -- start 2024-04-22T05:50:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 -- start start 2024-04-22T05:50:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6740047d0 0x7fe68c1da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe68c1daa20 0x7fe68c0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe68c1daea0 0x7fe68c0b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c136080 con 0x7fe68c1daa20 2024-04-22T05:50:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c0b1510 con 0x7fe6740047d0 2024-04-22T05:50:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c0b17b0 con 0x7fe68c1daea0 2024-04-22T05:50:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe68bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6740047d0 0x7fe68c1da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe68bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6740047d0 0x7fe68c1da4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38482/0 (socket says 172.21.15.62:38482) 2024-04-22T05:50:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe68bfff640 1 -- 172.21.15.62:0/1556085783 learned_addr learned my addr 172.21.15.62:0/1556085783 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-22T05:50:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe68b7fe640 1 --2- 172.21.15.62:0/1556085783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe68c1daa20 0x7fe68c0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe690df3640 1 --2- 172.21.15.62:0/1556085783 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe68c1daea0 0x7fe68c0b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe68bfff640 1 -- 172.21.15.62:0/1556085783 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe68c1daea0 msgr2=0x7fe68c0b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe68bfff640 1 --2- 172.21.15.62:0/1556085783 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe68c1daea0 0x7fe68c0b2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe68bfff640 1 -- 172.21.15.62:0/1556085783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe68c1daa20 msgr2=0x7fe68c0b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe68bfff640 1 --2- 172.21.15.62:0/1556085783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe68c1daa20 0x7fe68c0b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe68bfff640 1 -- 172.21.15.62:0/1556085783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c0b26c0 con 0x7fe6740047d0 2024-04-22T05:50:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe68bfff640 1 --2- 172.21.15.62:0/1556085783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6740047d0 0x7fe68c1da4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00e4d0 tx=0x7fe67c00e990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe688ff9640 1 -- 172.21.15.62:0/1556085783 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c0052e0 con 0x7fe6740047d0 2024-04-22T05:50:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe688ff9640 1 -- 172.21.15.62:0/1556085783 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c005440 con 0x7fe6740047d0 2024-04-22T05:50:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe688ff9640 1 -- 172.21.15.62:0/1556085783 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c003820 con 0x7fe6740047d0 2024-04-22T05:50:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 -- 172.21.15.62:0/1556085783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68c0b2910 con 0x7fe6740047d0 2024-04-22T05:50:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 -- 172.21.15.62:0/1556085783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68c133ea0 con 0x7fe6740047d0 2024-04-22T05:50:30.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe692856640 1 -- 172.21.15.62:0/1556085783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68c13cb90 con 0x7fe6740047d0 2024-04-22T05:50:30.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe688ff9640 1 -- 172.21.15.62:0/1556085783 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fe67c003c80 con 0x7fe6740047d0 2024-04-22T05:50:30.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.358+0000 7fe688ff9640 1 -- 172.21.15.62:0/1556085783 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fe67c027a90 con 0x7fe6740047d0 2024-04-22T05:50:30.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.362+0000 7fe688ff9640 1 -- 172.21.15.62:0/1556085783 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fe68c13cb90 con 0x7fe6740047d0 2024-04-22T05:50:30.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.430+0000 7fe692856640 1 -- 172.21.15.62:0/1556085783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d17323e1-9cec-4bcb-b603-7f9fda35b5f3", "id": 7} v 0) v1 -- 0x7fe68c066d70 con 0x7fe6740047d0 2024-04-22T05:50:30.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.434+0000 7fe688ff9640 1 -- 172.21.15.62:0/1556085783 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d17323e1-9cec-4bcb-b603-7f9fda35b5f3", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe67c027c50 con 0x7fe6740047d0 2024-04-22T05:50:30.437 INFO:teuthology.orchestra.run.smithi062.stdout:7 2024-04-22T05:50:30.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.438+0000 7fe692856640 1 -- 172.21.15.62:0/1556085783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6740047d0 msgr2=0x7fe68c1da4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.438+0000 7fe692856640 1 --2- 172.21.15.62:0/1556085783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6740047d0 0x7fe68c1da4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00e4d0 tx=0x7fe67c00e990 comp rx=0 tx=0).stop 2024-04-22T05:50:30.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.438+0000 7fe692856640 1 -- 172.21.15.62:0/1556085783 shutdown_connections 2024-04-22T05:50:30.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.438+0000 7fe692856640 1 --2- 172.21.15.62:0/1556085783 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe68c1daea0 0x7fe68c0b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.438+0000 7fe692856640 1 --2- 172.21.15.62:0/1556085783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe68c1daa20 0x7fe68c0b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.438+0000 7fe692856640 1 --2- 172.21.15.62:0/1556085783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6740047d0 0x7fe68c1da4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.438+0000 7fe692856640 1 -- 172.21.15.62:0/1556085783 >> 172.21.15.62:0/1556085783 conn(0x7fe68c0b3920 msgr2=0x7fe68c13a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:30.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.438+0000 7fe692856640 1 -- 172.21.15.62:0/1556085783 shutdown_connections 2024-04-22T05:50:30.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-22T05:50:30.438+0000 7fe692856640 1 -- 172.21.15.62:0/1556085783 wait complete. 2024-04-22T05:50:30.453 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-22T05:50:30.453 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-22T05:50:30.456 INFO:tasks.ceph.osd.0:Started 2024-04-22T05:50:30.456 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-22T05:50:30.456 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-22T05:50:30.458 INFO:tasks.ceph.osd.1:Started 2024-04-22T05:50:30.458 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-22T05:50:30.458 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-22T05:50:30.460 INFO:tasks.ceph.osd.2:Started 2024-04-22T05:50:30.461 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-22T05:50:30.461 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-22T05:50:30.463 INFO:tasks.ceph.osd.3:Started 2024-04-22T05:50:30.463 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-22T05:50:30.463 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-22T05:50:30.498 INFO:tasks.ceph.osd.4:Started 2024-04-22T05:50:30.499 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-22T05:50:30.499 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-22T05:50:30.501 INFO:tasks.ceph.osd.5:Started 2024-04-22T05:50:30.501 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-22T05:50:30.501 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-22T05:50:30.503 INFO:tasks.ceph.osd.6:Started 2024-04-22T05:50:30.503 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-22T05:50:30.503 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-22T05:50:30.505 INFO:tasks.ceph.osd.7:Started 2024-04-22T05:50:30.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T05:50:30.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.618+0000 7f4c51946640 1 Processor -- start 2024-04-22T05:50:30.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.618+0000 7f4c51946640 1 -- start start 2024-04-22T05:50:30.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.618+0000 7f4c51946640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4c4c0abca0 0x7f4c4c1444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.618+0000 7f4c51946640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c4c06a250 0x7f4c4c14b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.618+0000 7f4c51946640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4c0ad890 con 0x7f4c4c069e70 2024-04-22T05:50:30.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.618+0000 7f4c51946640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4c0acfc0 con 0x7f4c4c0abca0 2024-04-22T05:50:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.618+0000 7f4c51946640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c0b00b0 con 0x7f4c4c06a250 2024-04-22T05:50:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.618+0000 7f4c4affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4c4c069e70 0x7f4c4c0af560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33746/0 (socket says 172.21.15.40:33746) 2024-04-22T05:50:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.618+0000 7f4c4affd640 1 -- 172.21.15.40:0/987288877 learned_addr learned my addr 172.21.15.40:0/987288877 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:50:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.618+0000 7f4c4b7fe640 1 --2- 172.21.15.40:0/987288877 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c4c06a250 0x7f4c4c14b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.618+0000 7f4c4a7fc640 1 --2- 172.21.15.40:0/987288877 >> v2:172.21.15.62:3300/0 conn(0x7f4c4c0abca0 0x7f4c4c1444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.618+0000 7f4c49ffb640 1 -- 172.21.15.40:0/987288877 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672520278 0 0) 0x7f4c4c0ad890 con 0x7f4c4c069e70 2024-04-22T05:50:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c49ffb640 1 -- 172.21.15.40:0/987288877 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c34003530 con 0x7f4c4c069e70 2024-04-22T05:50:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c4b7fe640 1 -- 172.21.15.40:0/987288877 >> v2:172.21.15.62:3300/0 conn(0x7f4c4c0abca0 msgr2=0x7f4c4c1444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c4b7fe640 1 --2- 172.21.15.40:0/987288877 >> v2:172.21.15.62:3300/0 conn(0x7f4c4c0abca0 0x7f4c4c1444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c4b7fe640 1 -- 172.21.15.40:0/987288877 >> v1:172.21.15.40:6789/0 conn(0x7f4c4c069e70 legacy=0x7f4c4c0af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c4b7fe640 1 -- 172.21.15.40:0/987288877 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c0b01f0 con 0x7f4c4c06a250 2024-04-22T05:50:30.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c4b7fe640 1 --2- 172.21.15.40:0/987288877 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c4c06a250 0x7f4c4c14b830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4c440097c0 tx=0x7f4c4401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5481a956c758a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:30.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c49ffb640 1 -- 172.21.15.40:0/987288877 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c440ab610 con 0x7f4c4c06a250 2024-04-22T05:50:30.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c49ffb640 1 -- 172.21.15.40:0/987288877 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c440ab770 con 0x7f4c4c06a250 2024-04-22T05:50:30.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c49ffb640 1 -- 172.21.15.40:0/987288877 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c440abb00 con 0x7f4c4c06a250 2024-04-22T05:50:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c51946640 1 -- 172.21.15.40:0/987288877 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c4c06a250 msgr2=0x7f4c4c14b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c51946640 1 --2- 172.21.15.40:0/987288877 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c4c06a250 0x7f4c4c14b830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4c440097c0 tx=0x7f4c4401c3f0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c51946640 1 -- 172.21.15.40:0/987288877 shutdown_connections 2024-04-22T05:50:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c51946640 1 --2- 172.21.15.40:0/987288877 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c4c06a250 0x7f4c4c14b830 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c51946640 1 --2- 172.21.15.40:0/987288877 >> v2:172.21.15.62:3300/0 conn(0x7f4c4c0abca0 0x7f4c4c1444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c51946640 1 -- 172.21.15.40:0/987288877 >> 172.21.15.40:0/987288877 conn(0x7f4c4c0b3920 msgr2=0x7f4c4c0b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c51946640 1 -- 172.21.15.40:0/987288877 shutdown_connections 2024-04-22T05:50:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.622+0000 7f4c51946640 1 -- 172.21.15.40:0/987288877 wait complete. 2024-04-22T05:50:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c51946640 1 Processor -- start 2024-04-22T05:50:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c51946640 1 -- start start 2024-04-22T05:50:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c51946640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c4c069e70 0x7f4c4c1418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c51946640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c4c0abca0 0x7f4c4c13ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c51946640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c4c1d91c0 0x7f4c4c1d9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c51946640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c140e60 con 0x7f4c4c1d91c0 2024-04-22T05:50:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c51946640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c143fc0 con 0x7f4c4c069e70 2024-04-22T05:50:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c51946640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c1442a0 con 0x7f4c4c0abca0 2024-04-22T05:50:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c4a7fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c4c0abca0 0x7f4c4c13ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c4b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c4c1d91c0 0x7f4c4c1d9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c4a7fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c4c0abca0 0x7f4c4c13ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:37678/0 (socket says 172.21.15.40:37678) 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c4a7fc640 1 -- 172.21.15.40:0/916294808 learned_addr learned my addr 172.21.15.40:0/916294808 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c4affd640 1 --2- 172.21.15.40:0/916294808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c4c069e70 0x7f4c4c1418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c4a7fc640 1 -- 172.21.15.40:0/916294808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c4c069e70 msgr2=0x7f4c4c1418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c4a7fc640 1 --2- 172.21.15.40:0/916294808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c4c069e70 0x7f4c4c1418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c4a7fc640 1 -- 172.21.15.40:0/916294808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c4c1d91c0 msgr2=0x7f4c4c1d9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c4a7fc640 1 --2- 172.21.15.40:0/916294808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c4c1d91c0 0x7f4c4c1d9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c4a7fc640 1 -- 172.21.15.40:0/916294808 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c44009420 con 0x7f4c4c0abca0 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c4b7fe640 1 --2- 172.21.15.40:0/916294808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c4c1d91c0 0x7f4c4c1d9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c4a7fc640 1 --2- 172.21.15.40:0/916294808 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c4c0abca0 0x7f4c4c13ab90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000efc0 tx=0x7f4c4000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c33fff640 1 -- 172.21.15.40:0/916294808 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40009b80 con 0x7f4c4c0abca0 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c33fff640 1 -- 172.21.15.40:0/916294808 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4000f040 con 0x7f4c4c0abca0 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c51946640 1 -- 172.21.15.40:0/916294808 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c13b0d0 con 0x7f4c4c0abca0 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c33fff640 1 -- 172.21.15.40:0/916294808 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c400146a0 con 0x7f4c4c0abca0 2024-04-22T05:50:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c51946640 1 -- 172.21.15.40:0/916294808 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c1d9d50 con 0x7f4c4c0abca0 2024-04-22T05:50:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c33fff640 1 -- 172.21.15.40:0/916294808 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f4c40014b00 con 0x7f4c4c0abca0 2024-04-22T05:50:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c33fff640 1 -- 172.21.15.40:0/916294808 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4c40013ab0 con 0x7f4c4c0abca0 2024-04-22T05:50:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.626+0000 7f4c51946640 1 -- 172.21.15.40:0/916294808 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4c144460 con 0x7f4c4c0abca0 2024-04-22T05:50:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.630+0000 7f4c33fff640 1 -- 172.21.15.40:0/916294808 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f4c40018020 con 0x7f4c4c0abca0 2024-04-22T05:50:30.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.718+0000 7f4c51946640 1 -- 172.21.15.40:0/916294808 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c4c066d70 con 0x7f4c4c0abca0 2024-04-22T05:50:30.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.718+0000 7f4c33fff640 1 -- 172.21.15.40:0/916294808 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4c40013c70 con 0x7f4c4c0abca0 2024-04-22T05:50:30.720 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:50:30.720 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":10,"fsid":"36e9f223-2858-4559-8f74-0be5d30b8b0d","created":"2024-04-22T05:50:28.068495+0000","modified":"2024-04-22T05:50:30.434344+0000","last_up_change":"0.000000","last_in_change":"2024-04-22T05:50:30.434344+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1997ac14-110c-4d7e-8ac5-d0ac4f70af75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"cb6b2995-f7ce-4a68-ac16-d1690cc2b9ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a5634ce0-a08d-403f-bb05-eadb21417851","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bada948d-c18e-4bf2-8902-9f383fcd1b26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4ad76cfa-9e85-421e-bd2e-61006dd694bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b3b6f5a2-d949-429c-bfc4-b6a7333d8184","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00c5caf5-710a-4162-93fa-917a4823ffaf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d17323e1-9cec-4bcb-b603-7f9fda35b5f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T05:50:30.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.718+0000 7f4c31ffb640 1 -- 172.21.15.40:0/916294808 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c4c0abca0 msgr2=0x7f4c4c13ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:30.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.718+0000 7f4c31ffb640 1 --2- 172.21.15.40:0/916294808 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c4c0abca0 0x7f4c4c13ab90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000efc0 tx=0x7f4c4000c490 comp rx=0 tx=0).stop 2024-04-22T05:50:30.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.718+0000 7f4c31ffb640 1 -- 172.21.15.40:0/916294808 shutdown_connections 2024-04-22T05:50:30.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.718+0000 7f4c31ffb640 1 --2- 172.21.15.40:0/916294808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c4c1d91c0 0x7f4c4c1d9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.718+0000 7f4c31ffb640 1 --2- 172.21.15.40:0/916294808 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c4c0abca0 0x7f4c4c13ab90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.718+0000 7f4c31ffb640 1 --2- 172.21.15.40:0/916294808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c4c069e70 0x7f4c4c1418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:30.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.718+0000 7f4c31ffb640 1 -- 172.21.15.40:0/916294808 >> 172.21.15.40:0/916294808 conn(0x7f4c4c0b3920 msgr2=0x7f4c4c0af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:30.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.718+0000 7f4c31ffb640 1 -- 172.21.15.40:0/916294808 shutdown_connections 2024-04-22T05:50:30.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:30.718+0000 7f4c31ffb640 1 -- 172.21.15.40:0/916294808 wait complete. 2024-04-22T05:50:30.745 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-22T05:50:30.745 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-22T05:50:30.944 INFO:tasks.ceph.mgr.x.smithi062.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-22T05:50:30.944 INFO:tasks.ceph.mgr.x.smithi062.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-22T05:50:30.944 INFO:tasks.ceph.mgr.x.smithi062.stderr: from numpy import show_config as show_numpy_config 2024-04-22T05:50:30.993 INFO:tasks.ceph.mgr.y.smithi040.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-22T05:50:30.993 INFO:tasks.ceph.mgr.y.smithi040.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-22T05:50:30.993 INFO:tasks.ceph.mgr.y.smithi040.stderr: from numpy import show_config as show_numpy_config 2024-04-22T05:50:31.012 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:31.010+0000 7f509a89e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T05:50:31.069 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:31.066+0000 7f7ca365d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T05:50:31.075 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-22T05:50:31.074+0000 7fb58c4697c0 -1 Falling back to public interface 2024-04-22T05:50:31.077 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-22T05:50:31.074+0000 7f3c285827c0 -1 Falling back to public interface 2024-04-22T05:50:31.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-22T05:50:31.090+0000 7f7043d4d7c0 -1 Falling back to public interface 2024-04-22T05:50:31.091 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-22T05:50:31.090+0000 7ff49b6cf7c0 -1 Falling back to public interface 2024-04-22T05:50:31.131 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-04-22T05:50:31.130+0000 7f20b4a0b7c0 -1 Falling back to public interface 2024-04-22T05:50:31.134 INFO:tasks.ceph.osd.7.smithi062.stderr:2024-04-22T05:50:31.134+0000 7f7c86cd67c0 -1 Falling back to public interface 2024-04-22T05:50:31.142 INFO:tasks.ceph.osd.4.smithi062.stderr:2024-04-22T05:50:31.142+0000 7f4114f677c0 -1 Falling back to public interface 2024-04-22T05:50:31.146 INFO:tasks.ceph.osd.5.smithi062.stderr:2024-04-22T05:50:31.146+0000 7f84202737c0 -1 Falling back to public interface 2024-04-22T05:50:31.202 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:31.202+0000 7f509a89e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T05:50:31.254 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:31.250+0000 7f7ca365d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T05:50:31.355 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:31.354+0000 7f509a89e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T05:50:31.416 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:31.414+0000 7f7ca365d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T05:50:31.420 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:31.418+0000 7f509a89e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T05:50:31.485 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:31.478+0000 7f7ca365d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T05:50:31.551 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-22T05:50:31.560 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:31.558+0000 7f509a89e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T05:50:31.622 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:31.618+0000 7f7ca365d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T05:50:31.670 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.666+0000 7fcb4e902640 1 Processor -- start 2024-04-22T05:50:31.670 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb4e902640 1 -- start start 2024-04-22T05:50:31.672 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb4e902640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcb4806c1f0 0x7fcb4813e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:31.672 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb4e902640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb480aedf0 0x7fcb4813e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:31.672 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb4e902640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4806b4b0 con 0x7fcb4806c5d0 2024-04-22T05:50:31.672 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb4e902640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4813ecb0 con 0x7fcb480aedf0 2024-04-22T05:50:31.672 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb4e902640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4806b790 con 0x7fcb4806c1f0 2024-04-22T05:50:31.672 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb477fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb480aedf0 0x7fcb4813e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:31.672 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb477fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb480aedf0 0x7fcb4813e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34218/0 (socket says 172.21.15.40:34218) 2024-04-22T05:50:31.672 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb477fe640 1 -- 172.21.15.40:0/3896487615 learned_addr learned my addr 172.21.15.40:0/3896487615 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:31.672 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb47fff640 1 --2- 172.21.15.40:0/3896487615 >> v2:172.21.15.62:3300/0 conn(0x7fcb4806c1f0 0x7fcb4813e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:31.672 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb477fe640 1 -- 172.21.15.40:0/3896487615 >> v2:172.21.15.62:3300/0 conn(0x7fcb4806c1f0 msgr2=0x7fcb4813e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:31.672 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb477fe640 1 --2- 172.21.15.40:0/3896487615 >> v2:172.21.15.62:3300/0 conn(0x7fcb4806c1f0 0x7fcb4813e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:31.673 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb477fe640 1 -- 172.21.15.40:0/3896487615 >> v1:172.21.15.62:6789/0 conn(0x7fcb4806c5d0 legacy=0x7fcb481447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:31.673 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb477fe640 1 -- 172.21.15.40:0/3896487615 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4806cb00 con 0x7fcb480aedf0 2024-04-22T05:50:31.675 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb477fe640 1 --2- 172.21.15.40:0/3896487615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480aedf0 0x7fcb4813e770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcb480b2560 tx=0x7fcb3801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128483ada9b02d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:31.675 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb46ffd640 1 -- 172.21.15.40:0/3896487615 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb380ab650 con 0x7fcb480aedf0 2024-04-22T05:50:31.675 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb46ffd640 1 -- 172.21.15.40:0/3896487615 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb380ab7b0 con 0x7fcb480aedf0 2024-04-22T05:50:31.675 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.670+0000 7fcb46ffd640 1 -- 172.21.15.40:0/3896487615 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb380abb40 con 0x7fcb480aedf0 2024-04-22T05:50:31.675 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 -- 172.21.15.40:0/3896487615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480aedf0 msgr2=0x7fcb4813e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:31.675 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 --2- 172.21.15.40:0/3896487615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480aedf0 0x7fcb4813e770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcb480b2560 tx=0x7fcb3801c430 comp rx=0 tx=0).stop 2024-04-22T05:50:31.675 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 -- 172.21.15.40:0/3896487615 shutdown_connections 2024-04-22T05:50:31.675 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 --2- 172.21.15.40:0/3896487615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480aedf0 0x7fcb4813e770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:31.675 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 --2- 172.21.15.40:0/3896487615 >> v2:172.21.15.62:3300/0 conn(0x7fcb4806c1f0 0x7fcb4813e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:31.676 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 -- 172.21.15.40:0/3896487615 >> 172.21.15.40:0/3896487615 conn(0x7fcb48073b10 msgr2=0x7fcb480b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:31.676 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 -- 172.21.15.40:0/3896487615 shutdown_connections 2024-04-22T05:50:31.676 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 -- 172.21.15.40:0/3896487615 wait complete. 2024-04-22T05:50:31.679 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 Processor -- start 2024-04-22T05:50:31.679 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 -- start start 2024-04-22T05:50:31.679 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4806c1f0 0x7fcb481361e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:31.679 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fcb4806c5d0 0x7fcb481dc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:31.679 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480aedf0 0x7fcb481dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:31.679 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb481dc3c0 con 0x7fcb480aedf0 2024-04-22T05:50:31.679 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb481345d0 con 0x7fcb4806c1f0 2024-04-22T05:50:31.679 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb48132a50 con 0x7fcb4806c5d0 2024-04-22T05:50:31.679 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb477fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fcb4806c5d0 0x7fcb481dc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:31.679 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb477fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fcb4806c5d0 0x7fcb481dc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:37692/0 (socket says 172.21.15.40:37692) 2024-04-22T05:50:31.679 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb477fe640 1 -- 172.21.15.40:0/3064355566 learned_addr learned my addr 172.21.15.40:0/3064355566 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:31.680 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4ce9f640 1 --2- 172.21.15.40:0/3064355566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480aedf0 0x7fcb481dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:31.680 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb47fff640 1 --2- 172.21.15.40:0/3064355566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4806c1f0 0x7fcb481361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:31.680 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb47fff640 1 -- 172.21.15.40:0/3064355566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4806c1f0 msgr2=0x7fcb481361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-22T05:50:31.680 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb47fff640 1 -- 172.21.15.40:0/3064355566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4806c1f0 msgr2=0x7fcb481361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T05:50:31.680 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb47fff640 1 --2- 172.21.15.40:0/3064355566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4806c1f0 0x7fcb481361e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:31.680 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb47fff640 1 --2- 172.21.15.40:0/3064355566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4806c1f0 0x7fcb481361e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T05:50:31.680 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb477fe640 1 -- 172.21.15.40:0/3064355566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4806c1f0 msgr2=0x7fcb481361e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T05:50:31.680 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb477fe640 1 --2- 172.21.15.40:0/3064355566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4806c1f0 0x7fcb481361e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:31.680 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb477fe640 1 -- 172.21.15.40:0/3064355566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480aedf0 msgr2=0x7fcb481dcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:31.680 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb477fe640 1 --2- 172.21.15.40:0/3064355566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480aedf0 0x7fcb481dcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:31.680 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb477fe640 1 -- 172.21.15.40:0/3064355566 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb380094d0 con 0x7fcb4806c5d0 2024-04-22T05:50:31.680 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb477fe640 1 --2- 172.21.15.40:0/3064355566 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fcb4806c5d0 0x7fcb481dc7e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb3801c910 tx=0x7fcb3801ac70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:31.681 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb44ff9640 1 -- 172.21.15.40:0/3064355566 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb380aabf0 con 0x7fcb4806c5d0 2024-04-22T05:50:31.681 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb44ff9640 1 -- 172.21.15.40:0/3064355566 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb380aad70 con 0x7fcb4806c5d0 2024-04-22T05:50:31.681 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 -- 172.21.15.40:0/3064355566 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb48135b10 con 0x7fcb4806c5d0 2024-04-22T05:50:31.681 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb44ff9640 1 -- 172.21.15.40:0/3064355566 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb380be630 con 0x7fcb4806c5d0 2024-04-22T05:50:31.681 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb4e902640 1 -- 172.21.15.40:0/3064355566 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb481368f0 con 0x7fcb4806c5d0 2024-04-22T05:50:31.681 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb44ff9640 1 -- 172.21.15.40:0/3064355566 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fcb380bea90 con 0x7fcb4806c5d0 2024-04-22T05:50:31.681 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.674+0000 7fcb44ff9640 1 -- 172.21.15.40:0/3064355566 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fcb380b43f0 con 0x7fcb4806c5d0 2024-04-22T05:50:31.681 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.678+0000 7fcb4e902640 1 -- 172.21.15.40:0/3064355566 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb481dc500 con 0x7fcb4806c5d0 2024-04-22T05:50:31.682 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.678+0000 7fcb44ff9640 1 -- 172.21.15.40:0/3064355566 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fcb481dc500 con 0x7fcb4806c5d0 2024-04-22T05:50:31.749 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.746+0000 7fcb4e902640 1 -- 172.21.15.40:0/3064355566 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb48136ae0 con 0x7fcb4806c5d0 2024-04-22T05:50:31.750 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.750+0000 7fcb44ff9640 1 -- 172.21.15.40:0/3064355566 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fcb380c7440 con 0x7fcb4806c5d0 2024-04-22T05:50:31.750 INFO:teuthology.misc.health.smithi040.stdout: 2024-04-22T05:50:31.751 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":10,"fsid":"36e9f223-2858-4559-8f74-0be5d30b8b0d","created":"2024-04-22T05:50:28.068495+0000","modified":"2024-04-22T05:50:30.434344+0000","last_up_change":"0.000000","last_in_change":"2024-04-22T05:50:30.434344+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1997ac14-110c-4d7e-8ac5-d0ac4f70af75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"cb6b2995-f7ce-4a68-ac16-d1690cc2b9ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a5634ce0-a08d-403f-bb05-eadb21417851","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bada948d-c18e-4bf2-8902-9f383fcd1b26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4ad76cfa-9e85-421e-bd2e-61006dd694bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b3b6f5a2-d949-429c-bfc4-b6a7333d8184","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00c5caf5-710a-4162-93fa-917a4823ffaf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d17323e1-9cec-4bcb-b603-7f9fda35b5f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T05:50:31.751 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.750+0000 7fcb4e902640 1 -- 172.21.15.40:0/3064355566 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fcb4806c5d0 msgr2=0x7fcb481dc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:31.751 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.750+0000 7fcb4e902640 1 --2- 172.21.15.40:0/3064355566 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fcb4806c5d0 0x7fcb481dc7e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb3801c910 tx=0x7fcb3801ac70 comp rx=0 tx=0).stop 2024-04-22T05:50:31.751 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.750+0000 7fcb4e902640 1 -- 172.21.15.40:0/3064355566 shutdown_connections 2024-04-22T05:50:31.752 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.750+0000 7fcb4e902640 1 --2- 172.21.15.40:0/3064355566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb480aedf0 0x7fcb481dcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:31.752 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.750+0000 7fcb4e902640 1 --2- 172.21.15.40:0/3064355566 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fcb4806c5d0 0x7fcb481dc7e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:31.752 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.750+0000 7fcb4e902640 1 --2- 172.21.15.40:0/3064355566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4806c1f0 0x7fcb481361e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:31.752 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.750+0000 7fcb4e902640 1 -- 172.21.15.40:0/3064355566 >> 172.21.15.40:0/3064355566 conn(0x7fcb48073b10 msgr2=0x7fcb480b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:31.752 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.750+0000 7fcb4e902640 1 -- 172.21.15.40:0/3064355566 shutdown_connections 2024-04-22T05:50:31.752 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:31.750+0000 7fcb4e902640 1 -- 172.21.15.40:0/3064355566 wait complete. 2024-04-22T05:50:31.767 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-22T05:50:32.195 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:32.194+0000 7f509a89e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T05:50:32.269 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:32.266+0000 7f7ca365d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T05:50:32.273 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:32.274+0000 7f509a89e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T05:50:32.347 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:32.346+0000 7f7ca365d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T05:50:32.461 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:32.458+0000 7f509a89e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T05:50:32.534 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:32.534+0000 7f7ca365d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T05:50:32.538 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:32.534+0000 7f509a89e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T05:50:32.611 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:32.610+0000 7f7ca365d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T05:50:32.838 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:32.838+0000 7f509a89e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T05:50:32.902 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:32.901+0000 7f509a89e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T05:50:32.912 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:32.910+0000 7f7ca365d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T05:50:32.980 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-22T05:50:32.978+0000 7f7ca365d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T05:50:33.941 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:33.941+0000 7f5041e52640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-22T05:50:33.942 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-22T05:50:33.941+0000 7f5041e52640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-22T05:50:34.445 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-22T05:50:34.442+0000 7ff49b6cf7c0 -1 osd.3 0 log_to_monitors true 2024-04-22T05:50:34.924 INFO:tasks.ceph.osd.7.smithi062.stderr:2024-04-22T05:50:34.921+0000 7f7c86cd67c0 -1 osd.7 0 log_to_monitors true 2024-04-22T05:50:34.925 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-22T05:50:34.922+0000 7fb58c4697c0 -1 osd.1 0 log_to_monitors true 2024-04-22T05:50:34.930 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-22T05:50:34.930+0000 7f7043d4d7c0 -1 osd.0 0 log_to_monitors true 2024-04-22T05:50:34.944 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-04-22T05:50:34.941+0000 7f20b4a0b7c0 -1 osd.6 0 log_to_monitors true 2024-04-22T05:50:34.960 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-22T05:50:34.958+0000 7f3c285827c0 -1 osd.2 0 log_to_monitors true 2024-04-22T05:50:35.038 INFO:tasks.ceph.osd.4.smithi062.stderr:2024-04-22T05:50:35.037+0000 7f4114f677c0 -1 osd.4 0 log_to_monitors true 2024-04-22T05:50:35.057 INFO:tasks.ceph.osd.5.smithi062.stderr:2024-04-22T05:50:35.057+0000 7f84202737c0 -1 osd.5 0 log_to_monitors true 2024-04-22T05:50:35.974 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-22T05:50:35.974+0000 7fb58840e640 -1 osd.1 0 waiting for initial osdmap 2024-04-22T05:50:35.999 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-22T05:50:35.998+0000 7fb584249640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T05:50:36.679 INFO:tasks.ceph.osd.7.smithi062.stderr:2024-04-22T05:50:36.677+0000 7f7c82c7b640 -1 osd.7 0 waiting for initial osdmap 2024-04-22T05:50:36.695 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-22T05:50:36.694+0000 7ff497674640 -1 osd.3 0 waiting for initial osdmap 2024-04-22T05:50:36.708 INFO:tasks.ceph.osd.7.smithi062.stderr:2024-04-22T05:50:36.705+0000 7f7c82c7b640 -1 osd.7 0 waiting for initial osdmap 2024-04-22T05:50:36.710 INFO:tasks.ceph.osd.7.smithi062.stderr:2024-04-22T05:50:36.709+0000 7f7c7aaae640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T05:50:36.718 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-22T05:50:36.718+0000 7ff497674640 -1 osd.3 0 waiting for initial osdmap 2024-04-22T05:50:36.721 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-22T05:50:36.718+0000 7ff48f4a7640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T05:50:36.983 INFO:tasks.ceph.osd.4.smithi062.stderr:2024-04-22T05:50:36.981+0000 7f4110f0c640 -1 osd.4 0 waiting for initial osdmap 2024-04-22T05:50:36.984 INFO:tasks.ceph.osd.5.smithi062.stderr:2024-04-22T05:50:36.981+0000 7f841c218640 -1 osd.5 0 waiting for initial osdmap 2024-04-22T05:50:36.984 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-04-22T05:50:36.981+0000 7f20b09b0640 -1 osd.6 0 waiting for initial osdmap 2024-04-22T05:50:37.006 INFO:tasks.ceph.osd.4.smithi062.stderr:2024-04-22T05:50:37.005+0000 7f410cd47640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T05:50:37.014 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-04-22T05:50:37.013+0000 7f20ac7eb640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T05:50:37.015 INFO:tasks.ceph.osd.5.smithi062.stderr:2024-04-22T05:50:37.013+0000 7f8418053640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T05:50:37.852 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-22T05:50:37.850+0000 7f3c24527640 -1 osd.2 0 waiting for initial osdmap 2024-04-22T05:50:37.871 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-22T05:50:37.870+0000 7f3c1c35a640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T05:50:37.872 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-22T05:50:37.870+0000 7f703fcf2640 -1 osd.0 0 waiting for initial osdmap 2024-04-22T05:50:37.899 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-22T05:50:37.898+0000 7f7037b25640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T05:50:38.575 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-22T05:50:38.700 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde2c9c0640 1 Processor -- start 2024-04-22T05:50:38.701 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde2c9c0640 1 -- start start 2024-04-22T05:50:38.702 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde2c9c0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fde2806eec0 0x7fde28142650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:38.702 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde2c9c0640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fde2806dc20 0x7fde28142b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:38.702 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde2c9c0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fde28070780 0x7fde28070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:38.702 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde2c9c0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde2806e2d0 con 0x7fde2806eec0 2024-04-22T05:50:38.702 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde2c9c0640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde2806f2a0 con 0x7fde28070780 2024-04-22T05:50:38.702 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde2c9c0640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde281430d0 con 0x7fde2806dc20 2024-04-22T05:50:38.702 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde25d9b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fde2806dc20 0x7fde28142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:38.702 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde25d9b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fde2806dc20 0x7fde28142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:37754/0 (socket says 172.21.15.40:37754) 2024-04-22T05:50:38.702 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde25d9b640 1 -- 172.21.15.40:0/287797778 learned_addr learned my addr 172.21.15.40:0/287797778 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:38.702 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde2659c640 1 --2- 172.21.15.40:0/287797778 >> v2:172.21.15.40:3300/0 conn(0x7fde2806eec0 0x7fde28142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:38.702 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde26d9d640 1 --2- 172.21.15.40:0/287797778 >> v2:172.21.15.62:3300/0 conn(0x7fde28070780 0x7fde28070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:38.703 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde25d9b640 1 -- 172.21.15.40:0/287797778 >> v2:172.21.15.62:3300/0 conn(0x7fde28070780 msgr2=0x7fde28070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:38.703 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde25d9b640 1 --2- 172.21.15.40:0/287797778 >> v2:172.21.15.62:3300/0 conn(0x7fde28070780 0x7fde28070bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:38.703 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde25d9b640 1 -- 172.21.15.40:0/287797778 >> v2:172.21.15.40:3300/0 conn(0x7fde2806eec0 msgr2=0x7fde28142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:38.703 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde25d9b640 1 --2- 172.21.15.40:0/287797778 >> v2:172.21.15.40:3300/0 conn(0x7fde2806eec0 0x7fde28142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:38.703 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.698+0000 7fde25d9b640 1 -- 172.21.15.40:0/287797778 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde28071940 con 0x7fde2806dc20 2024-04-22T05:50:38.704 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde25d9b640 1 --2- 172.21.15.40:0/287797778 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fde2806dc20 0x7fde28142b90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fde2806dbb0 tx=0x7fde1001c9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b747ecfd491302f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:38.704 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2559a640 1 -- 172.21.15.40:0/287797778 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde100a5760 con 0x7fde2806dc20 2024-04-22T05:50:38.704 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2559a640 1 -- 172.21.15.40:0/287797778 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde280ad030 con 0x7fde2806dc20 2024-04-22T05:50:38.704 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2559a640 1 -- 172.21.15.40:0/287797778 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde100a5ae0 con 0x7fde2806dc20 2024-04-22T05:50:38.705 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2c9c0640 1 -- 172.21.15.40:0/287797778 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fde2806dc20 msgr2=0x7fde28142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:38.705 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2c9c0640 1 --2- 172.21.15.40:0/287797778 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fde2806dc20 0x7fde28142b90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fde2806dbb0 tx=0x7fde1001c9d0 comp rx=0 tx=0).stop 2024-04-22T05:50:38.705 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2c9c0640 1 -- 172.21.15.40:0/287797778 shutdown_connections 2024-04-22T05:50:38.705 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2c9c0640 1 --2- 172.21.15.40:0/287797778 >> v2:172.21.15.62:3300/0 conn(0x7fde28070780 0x7fde28070bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:38.706 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2c9c0640 1 --2- 172.21.15.40:0/287797778 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fde2806dc20 0x7fde28142b90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:38.706 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2c9c0640 1 --2- 172.21.15.40:0/287797778 >> v2:172.21.15.40:3300/0 conn(0x7fde2806eec0 0x7fde28142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:38.706 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2c9c0640 1 -- 172.21.15.40:0/287797778 >> 172.21.15.40:0/287797778 conn(0x7fde2806a9f0 msgr2=0x7fde280b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:38.706 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2c9c0640 1 -- 172.21.15.40:0/287797778 shutdown_connections 2024-04-22T05:50:38.706 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2c9c0640 1 -- 172.21.15.40:0/287797778 wait complete. 2024-04-22T05:50:38.708 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2c9c0640 1 Processor -- start 2024-04-22T05:50:38.708 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.702+0000 7fde2c9c0640 1 -- start start 2024-04-22T05:50:38.708 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2c9c0640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fde2806dc20 0x7fde28147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:38.708 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2c9c0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde2806eec0 0x7fde28144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:38.708 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2c9c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde28070780 0x7fde281450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:38.708 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2c9c0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde28144740 con 0x7fde28070780 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2c9c0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde28143ca0 con 0x7fde2806eec0 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2c9c0640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde28143f80 con 0x7fde2806dc20 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2659c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fde2806dc20 0x7fde28147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde26d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde28070780 0x7fde281450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde26d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde28070780 0x7fde281450c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34264/0 (socket says 172.21.15.40:34264) 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde26d9d640 1 -- 172.21.15.40:0/4235389112 learned_addr learned my addr 172.21.15.40:0/4235389112 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde25d9b640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde2806eec0 0x7fde28144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2659c640 1 -- 172.21.15.40:0/4235389112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde2806eec0 msgr2=0x7fde28144b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2659c640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde2806eec0 0x7fde28144b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2659c640 1 -- 172.21.15.40:0/4235389112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde28070780 msgr2=0x7fde281450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2659c640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde28070780 0x7fde281450c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2659c640 1 -- 172.21.15.40:0/4235389112 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde10009b30 con 0x7fde2806dc20 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde26d9d640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde28070780 0x7fde281450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:50:38.709 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2659c640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fde2806dc20 0x7fde28147e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde1800e7f0 tx=0x7fde1800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:38.710 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde1effd640 1 -- 172.21.15.40:0/4235389112 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1800c940 con 0x7fde2806dc20 2024-04-22T05:50:38.710 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde1effd640 1 -- 172.21.15.40:0/4235389112 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1800caa0 con 0x7fde2806dc20 2024-04-22T05:50:38.710 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2c9c0640 1 -- 172.21.15.40:0/4235389112 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde28147180 con 0x7fde2806dc20 2024-04-22T05:50:38.710 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde1effd640 1 -- 172.21.15.40:0/4235389112 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1800ce10 con 0x7fde2806dc20 2024-04-22T05:50:38.710 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2c9c0640 1 -- 172.21.15.40:0/4235389112 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde28148520 con 0x7fde2806dc20 2024-04-22T05:50:38.710 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde2c9c0640 1 -- 172.21.15.40:0/4235389112 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde28067570 con 0x7fde2806dc20 2024-04-22T05:50:38.711 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde1effd640 1 -- 172.21.15.40:0/4235389112 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7fde18013470 con 0x7fde2806dc20 2024-04-22T05:50:38.711 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde1effd640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fde08066600 0x7fde08068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:38.711 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde1effd640 1 -- 172.21.15.40:0/4235389112 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fde18083cb0 con 0x7fde2806dc20 2024-04-22T05:50:38.711 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.706+0000 7fde25d9b640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fde08066600 0x7fde08068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:38.711 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.710+0000 7fde25d9b640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fde08066600 0x7fde08068ac0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde100ae000 tx=0x7fde100a7070 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:38.714 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.714+0000 7fde1effd640 1 -- 172.21.15.40:0/4235389112 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7fde18051670 con 0x7fde2806dc20 2024-04-22T05:50:38.876 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.874+0000 7fde2c9c0640 1 -- 172.21.15.40:0/4235389112 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde28147310 con 0x7fde2806dc20 2024-04-22T05:50:38.877 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.874+0000 7fde1effd640 1 -- 172.21.15.40:0/4235389112 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10461 (secure 0 0 0) 0x7fde18054c90 con 0x7fde2806dc20 2024-04-22T05:50:38.878 INFO:teuthology.misc.health.smithi040.stdout: 2024-04-22T05:50:38.878 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":14,"fsid":"36e9f223-2858-4559-8f74-0be5d30b8b0d","created":"2024-04-22T05:50:28.068495+0000","modified":"2024-04-22T05:50:37.978413+0000","last_up_change":"2024-04-22T05:50:37.978413+0000","last_in_change":"2024-04-22T05:50:30.434344+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1997ac14-110c-4d7e-8ac5-d0ac4f70af75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":24983},{"type":"v1","addr":"172.21.15.40:6818","nonce":24983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":24983},{"type":"v1","addr":"172.21.15.40:6822","nonce":24983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":24983},{"type":"v1","addr":"172.21.15.40:6830","nonce":24983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":24983},{"type":"v1","addr":"172.21.15.40:6826","nonce":24983}]},"public_addr":"172.21.15.40:6818/24983","cluster_addr":"172.21.15.40:6822/24983","heartbeat_back_addr":"172.21.15.40:6830/24983","heartbeat_front_addr":"172.21.15.40:6826/24983","state":["exists","up"]},{"osd":1,"uuid":"cb6b2995-f7ce-4a68-ac16-d1690cc2b9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":24984},{"type":"v1","addr":"172.21.15.40:6801","nonce":24984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":24984},{"type":"v1","addr":"172.21.15.40:6803","nonce":24984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":24984},{"type":"v1","addr":"172.21.15.40:6807","nonce":24984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":24984},{"type":"v1","addr":"172.21.15.40:6805","nonce":24984}]},"public_addr":"172.21.15.40:6801/24984","cluster_addr":"172.21.15.40:6803/24984","heartbeat_back_addr":"172.21.15.40:6807/24984","heartbeat_front_addr":"172.21.15.40:6805/24984","state":["exists","up"]},{"osd":2,"uuid":"a5634ce0-a08d-403f-bb05-eadb21417851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":24986},{"type":"v1","addr":"172.21.15.40:6809","nonce":24986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":24986},{"type":"v1","addr":"172.21.15.40:6811","nonce":24986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":24986},{"type":"v1","addr":"172.21.15.40:6815","nonce":24986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":24986},{"type":"v1","addr":"172.21.15.40:6813","nonce":24986}]},"public_addr":"172.21.15.40:6809/24986","cluster_addr":"172.21.15.40:6811/24986","heartbeat_back_addr":"172.21.15.40:6815/24986","heartbeat_front_addr":"172.21.15.40:6813/24986","state":["exists","up"]},{"osd":3,"uuid":"bada948d-c18e-4bf2-8902-9f383fcd1b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":24985},{"type":"v1","addr":"172.21.15.40:6819","nonce":24985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":24985},{"type":"v1","addr":"172.21.15.40:6823","nonce":24985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":24985},{"type":"v1","addr":"172.21.15.40:6831","nonce":24985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":24985},{"type":"v1","addr":"172.21.15.40:6827","nonce":24985}]},"public_addr":"172.21.15.40:6819/24985","cluster_addr":"172.21.15.40:6823/24985","heartbeat_back_addr":"172.21.15.40:6831/24985","heartbeat_front_addr":"172.21.15.40:6827/24985","state":["exists","up"]},{"osd":4,"uuid":"4ad76cfa-9e85-421e-bd2e-61006dd694bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":25135},{"type":"v1","addr":"172.21.15.62:6817","nonce":25135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":25135},{"type":"v1","addr":"172.21.15.62:6819","nonce":25135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":25135},{"type":"v1","addr":"172.21.15.62:6823","nonce":25135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":25135},{"type":"v1","addr":"172.21.15.62:6821","nonce":25135}]},"public_addr":"172.21.15.62:6817/25135","cluster_addr":"172.21.15.62:6819/25135","heartbeat_back_addr":"172.21.15.62:6823/25135","heartbeat_front_addr":"172.21.15.62:6821/25135","state":["exists","up"]},{"osd":5,"uuid":"b3b6f5a2-d949-429c-bfc4-b6a7333d8184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":25137},{"type":"v1","addr":"172.21.15.62:6825","nonce":25137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":25137},{"type":"v1","addr":"172.21.15.62:6827","nonce":25137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":25137},{"type":"v1","addr":"172.21.15.62:6831","nonce":25137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":25137},{"type":"v1","addr":"172.21.15.62:6829","nonce":25137}]},"public_addr":"172.21.15.62:6825/25137","cluster_addr":"172.21.15.62:6827/25137","heartbeat_back_addr":"172.21.15.62:6831/25137","heartbeat_front_addr":"172.21.15.62:6829/25137","state":["exists","up"]},{"osd":6,"uuid":"00c5caf5-710a-4162-93fa-917a4823ffaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":25136},{"type":"v1","addr":"172.21.15.62:6801","nonce":25136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":25136},{"type":"v1","addr":"172.21.15.62:6803","nonce":25136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":25136},{"type":"v1","addr":"172.21.15.62:6807","nonce":25136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":25136},{"type":"v1","addr":"172.21.15.62:6805","nonce":25136}]},"public_addr":"172.21.15.62:6801/25136","cluster_addr":"172.21.15.62:6803/25136","heartbeat_back_addr":"172.21.15.62:6807/25136","heartbeat_front_addr":"172.21.15.62:6805/25136","state":["exists","up"]},{"osd":7,"uuid":"d17323e1-9cec-4bcb-b603-7f9fda35b5f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":25138},{"type":"v1","addr":"172.21.15.62:6809","nonce":25138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":25138},{"type":"v1","addr":"172.21.15.62:6811","nonce":25138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":25138},{"type":"v1","addr":"172.21.15.62:6815","nonce":25138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":25138},{"type":"v1","addr":"172.21.15.62:6813","nonce":25138}]},"public_addr":"172.21.15.62:6809/25138","cluster_addr":"172.21.15.62:6811/25138","heartbeat_back_addr":"172.21.15.62:6815/25138","heartbeat_front_addr":"172.21.15.62:6813/25138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.944884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.419674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:36.683341+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T05:50:38.880 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.878+0000 7fde2c9c0640 1 -- 172.21.15.40:0/4235389112 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fde08066600 msgr2=0x7fde08068ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:38.880 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.878+0000 7fde2c9c0640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fde08066600 0x7fde08068ac0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde100ae000 tx=0x7fde100a7070 comp rx=0 tx=0).stop 2024-04-22T05:50:38.880 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.878+0000 7fde2c9c0640 1 -- 172.21.15.40:0/4235389112 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fde2806dc20 msgr2=0x7fde28147e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:38.881 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.878+0000 7fde2c9c0640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fde2806dc20 0x7fde28147e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde1800e7f0 tx=0x7fde1800ecb0 comp rx=0 tx=0).stop 2024-04-22T05:50:38.881 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.878+0000 7fde2c9c0640 1 -- 172.21.15.40:0/4235389112 shutdown_connections 2024-04-22T05:50:38.881 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.878+0000 7fde2c9c0640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fde08066600 0x7fde08068ac0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:38.881 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.878+0000 7fde2c9c0640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde2806eec0 0x7fde28144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:38.881 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.878+0000 7fde2c9c0640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde28070780 0x7fde281450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:38.881 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.878+0000 7fde2c9c0640 1 --2- 172.21.15.40:0/4235389112 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fde2806dc20 0x7fde28147e10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:38.881 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.878+0000 7fde2c9c0640 1 -- 172.21.15.40:0/4235389112 >> 172.21.15.40:0/4235389112 conn(0x7fde2806a9f0 msgr2=0x7fde28140970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:38.881 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.878+0000 7fde2c9c0640 1 -- 172.21.15.40:0/4235389112 shutdown_connections 2024-04-22T05:50:38.881 INFO:teuthology.misc.health.smithi040.stderr:2024-04-22T05:50:38.878+0000 7fde2c9c0640 1 -- 172.21.15.40:0/4235389112 wait complete. 2024-04-22T05:50:38.896 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-22T05:50:38.896 INFO:tasks.ceph:Creating RBD pool 2024-04-22T05:50:38.896 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-22T05:50:39.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.014+0000 7f7be1efe640 1 Processor -- start 2024-04-22T05:50:39.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.014+0000 7f7be1efe640 1 -- start start 2024-04-22T05:50:39.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.014+0000 7f7be1efe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7bdc073540 0x7f7bdc0aef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:39.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.014+0000 7f7be1efe640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bdc073920 con 0x7f7bdc0ae260 2024-04-22T05:50:39.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.014+0000 7f7be1efe640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bdc0a7b40 con 0x7f7bdc0ac740 2024-04-22T05:50:39.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.014+0000 7f7be1efe640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bdc0af4a0 con 0x7f7bdc073540 2024-04-22T05:50:39.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.014+0000 7f7bd3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7bdc0ae260 0x7f7bdc142280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33758/0 (socket says 172.21.15.40:33758) 2024-04-22T05:50:39.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.014+0000 7f7bd3fff640 1 -- 172.21.15.40:0/3198337394 learned_addr learned my addr 172.21.15.40:0/3198337394 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:50:39.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.018+0000 7f7bdb7fe640 1 --2- 172.21.15.40:0/3198337394 >> v2:172.21.15.62:3300/0 conn(0x7f7bdc073540 0x7f7bdc0aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:39.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.018+0000 7f7bdb7fe640 1 -- 172.21.15.40:0/3198337394 >> v1:172.21.15.62:6789/0 conn(0x7f7bdc0ac740 legacy=0x7f7bdc149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:39.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.018+0000 7f7bdb7fe640 1 -- 172.21.15.40:0/3198337394 >> v1:172.21.15.40:6789/0 conn(0x7f7bdc0ae260 legacy=0x7f7bdc142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:39.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.018+0000 7f7bdb7fe640 1 -- 172.21.15.40:0/3198337394 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bdc0a7d90 con 0x7f7bdc073540 2024-04-22T05:50:39.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.018+0000 7f7bdb7fe640 1 --2- 172.21.15.40:0/3198337394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bdc073540 0x7f7bdc0aef60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0097c0 tx=0x7f7bcc01c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e08d54b4bb369225 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:39.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.022+0000 7f7bdaffd640 1 -- 172.21.15.40:0/3198337394 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0ab600 con 0x7f7bdc073540 2024-04-22T05:50:39.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.022+0000 7f7bdaffd640 1 -- 172.21.15.40:0/3198337394 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bcc0a7050 con 0x7f7bdc073540 2024-04-22T05:50:39.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.022+0000 7f7bdaffd640 1 -- 172.21.15.40:0/3198337394 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0ab980 con 0x7f7bdc073540 2024-04-22T05:50:39.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 -- 172.21.15.40:0/3198337394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bdc073540 msgr2=0x7f7bdc0aef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:39.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 --2- 172.21.15.40:0/3198337394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bdc073540 0x7f7bdc0aef60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0097c0 tx=0x7f7bcc01c360 comp rx=0 tx=0).stop 2024-04-22T05:50:39.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 -- 172.21.15.40:0/3198337394 shutdown_connections 2024-04-22T05:50:39.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 --2- 172.21.15.40:0/3198337394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bdc073540 0x7f7bdc0aef60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:39.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 -- 172.21.15.40:0/3198337394 >> 172.21.15.40:0/3198337394 conn(0x7f7bdc0b3270 msgr2=0x7f7bdc0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:39.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 -- 172.21.15.40:0/3198337394 shutdown_connections 2024-04-22T05:50:39.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 -- 172.21.15.40:0/3198337394 wait complete. 2024-04-22T05:50:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 Processor -- start 2024-04-22T05:50:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 -- start start 2024-04-22T05:50:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7bdc073540 0x7f7bdc13ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bdc0ac740 0x7f7bdc1d9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bdc0ae260 0x7f7bdc1d95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc13e590 con 0x7f7bdc0ae260 2024-04-22T05:50:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc13cb00 con 0x7f7bdc0ac740 2024-04-22T05:50:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc13ccc0 con 0x7f7bdc073540 2024-04-22T05:50:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7bdc073540 0x7f7bdc13ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7bdc073540 0x7f7bdc13ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:37778/0 (socket says 172.21.15.40:37778) 2024-04-22T05:50:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bdb7fe640 1 -- 172.21.15.40:0/1080868083 learned_addr learned my addr 172.21.15.40:0/1080868083 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bdbfff640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bdc0ae260 0x7f7bdc1d95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bd3fff640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bdc0ac740 0x7f7bdc1d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bdb7fe640 1 -- 172.21.15.40:0/1080868083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bdc0ac740 msgr2=0x7f7bdc1d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bdb7fe640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bdc0ac740 0x7f7bdc1d9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bdb7fe640 1 -- 172.21.15.40:0/1080868083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bdc0ae260 msgr2=0x7f7bdc1d95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bdb7fe640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bdc0ae260 0x7f7bdc1d95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bdb7fe640 1 -- 172.21.15.40:0/1080868083 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc009420 con 0x7f7bdc073540 2024-04-22T05:50:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bd3fff640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bdc0ac740 0x7f7bdc1d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T05:50:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bdb7fe640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7bdc073540 0x7f7bdc13ddf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0ae040 tx=0x7f7bcc01aba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:39.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bd8ff9640 1 -- 172.21.15.40:0/1080868083 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0b7650 con 0x7f7bdc073540 2024-04-22T05:50:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bd8ff9640 1 -- 172.21.15.40:0/1080868083 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bcc0b77d0 con 0x7f7bdc073540 2024-04-22T05:50:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7bd8ff9640 1 -- 172.21.15.40:0/1080868083 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0b7b80 con 0x7f7bdc073540 2024-04-22T05:50:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 -- 172.21.15.40:0/1080868083 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bdc13fbe0 con 0x7f7bdc073540 2024-04-22T05:50:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.026+0000 7f7be1efe640 1 -- 172.21.15.40:0/1080868083 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc1d9ce0 con 0x7f7bdc073540 2024-04-22T05:50:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.030+0000 7f7be1efe640 1 -- 172.21.15.40:0/1080868083 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bdc066d70 con 0x7f7bdc073540 2024-04-22T05:50:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.030+0000 7f7bd8ff9640 1 -- 172.21.15.40:0/1080868083 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f7bcc0b7ce0 con 0x7f7bdc073540 2024-04-22T05:50:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.030+0000 7f7bd8ff9640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f7bbc066600 0x7f7bbc068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.030+0000 7f7bd8ff9640 1 -- 172.21.15.40:0/1080868083 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4846+0+0 (secure 0 0 0) 0x7f7bcc11e580 con 0x7f7bdc073540 2024-04-22T05:50:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.030+0000 7f7bd3fff640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f7bbc066600 0x7f7bbc068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:39.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.030+0000 7f7bd3fff640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f7bbc066600 0x7f7bbc068ac0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc1d6b30 tx=0x7f7bc4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:39.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.034+0000 7f7bd8ff9640 1 -- 172.21.15.40:0/1080868083 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f7bcc0ebd20 con 0x7f7bdc073540 2024-04-22T05:50:39.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:39.194+0000 7f7be1efe640 1 -- 172.21.15.40:0/1080868083 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f7bdc070580 con 0x7f7bdc073540 2024-04-22T05:50:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.030+0000 7f7bd8ff9640 1 -- 172.21.15.40:0/1080868083 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7f7bcc0b00a0 con 0x7f7bdc073540 2024-04-22T05:50:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'rbd' created 2024-04-22T05:50:40.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.034+0000 7f7be1efe640 1 -- 172.21.15.40:0/1080868083 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f7bbc066600 msgr2=0x7f7bbc068ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.034+0000 7f7be1efe640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f7bbc066600 0x7f7bbc068ac0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc1d6b30 tx=0x7f7bc4009040 comp rx=0 tx=0).stop 2024-04-22T05:50:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.034+0000 7f7be1efe640 1 -- 172.21.15.40:0/1080868083 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7bdc073540 msgr2=0x7f7bdc13ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.034+0000 7f7be1efe640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7bdc073540 0x7f7bdc13ddf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0ae040 tx=0x7f7bcc01aba0 comp rx=0 tx=0).stop 2024-04-22T05:50:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.034+0000 7f7be1efe640 1 -- 172.21.15.40:0/1080868083 shutdown_connections 2024-04-22T05:50:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.034+0000 7f7be1efe640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f7bbc066600 0x7f7bbc068ac0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.034+0000 7f7be1efe640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bdc0ac740 0x7f7bdc1d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.034+0000 7f7be1efe640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bdc0ae260 0x7f7bdc1d95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.034+0000 7f7be1efe640 1 --2- 172.21.15.40:0/1080868083 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7bdc073540 0x7f7bdc13ddf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.034+0000 7f7be1efe640 1 -- 172.21.15.40:0/1080868083 >> 172.21.15.40:0/1080868083 conn(0x7f7bdc0b3270 msgr2=0x7f7bdc0b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.034+0000 7f7be1efe640 1 -- 172.21.15.40:0/1080868083 shutdown_connections 2024-04-22T05:50:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.034+0000 7f7be1efe640 1 -- 172.21.15.40:0/1080868083 wait complete. 2024-04-22T05:50:40.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-22T05:50:40.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.194+0000 7f5ac9a6a640 1 Processor -- start 2024-04-22T05:50:40.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.194+0000 7f5ac9a6a640 1 -- start start 2024-04-22T05:50:40.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac9a6a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5ac4071250 0x7f5ac413c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:40.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac9a6a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac4071630 con 0x7f5ac40ae550 2024-04-22T05:50:40.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac9a6a640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac40a7bb0 con 0x7f5ac41429f0 2024-04-22T05:50:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac9a6a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac40a79d0 con 0x7f5ac4071250 2024-04-22T05:50:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac2ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5ac40ae550 0x7f5ac41422e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56002/0 (socket says 172.21.15.40:56002) 2024-04-22T05:50:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac2ffd640 1 -- 172.21.15.40:0/57001194 learned_addr learned my addr 172.21.15.40:0/57001194 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:50:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac37fe640 1 --2- 172.21.15.40:0/57001194 >> v2:172.21.15.62:3300/0 conn(0x7f5ac4071250 0x7f5ac413c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac27fc640 1 -- 172.21.15.40:0/57001194 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956139373 0 0) 0x7f5ac4071630 con 0x7f5ac40ae550 2024-04-22T05:50:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac27fc640 1 -- 172.21.15.40:0/57001194 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5aac003530 con 0x7f5ac40ae550 2024-04-22T05:50:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac27fc640 1 -- 172.21.15.40:0/57001194 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2549334368 0 0) 0x7f5aac003530 con 0x7f5ac40ae550 2024-04-22T05:50:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac37fe640 1 -- 172.21.15.40:0/57001194 >> v1:172.21.15.62:6789/0 conn(0x7f5ac41429f0 legacy=0x7f5ac4149630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac37fe640 1 -- 172.21.15.40:0/57001194 >> v1:172.21.15.40:6789/0 conn(0x7f5ac40ae550 legacy=0x7f5ac41422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac37fe640 1 -- 172.21.15.40:0/57001194 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac40a7e90 con 0x7f5ac4071250 2024-04-22T05:50:40.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.198+0000 7f5ac37fe640 1 --2- 172.21.15.40:0/57001194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac4071250 0x7f5ac413c010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40097c0 tx=0x7f5ab401cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8ddf2b0a35bd70c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:40.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac27fc640 1 -- 172.21.15.40:0/57001194 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8001680 con 0x7f5ac4071250 2024-04-22T05:50:40.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac27fc640 1 -- 172.21.15.40:0/57001194 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab40ac440 con 0x7f5ac4071250 2024-04-22T05:50:40.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac27fc640 1 -- 172.21.15.40:0/57001194 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40ac7b0 con 0x7f5ac4071250 2024-04-22T05:50:40.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/57001194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac4071250 msgr2=0x7f5ac413c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:40.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 --2- 172.21.15.40:0/57001194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac4071250 0x7f5ac413c010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40097c0 tx=0x7f5ab401cdd0 comp rx=0 tx=0).stop 2024-04-22T05:50:40.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/57001194 shutdown_connections 2024-04-22T05:50:40.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 --2- 172.21.15.40:0/57001194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac4071250 0x7f5ac413c010 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:40.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/57001194 >> 172.21.15.40:0/57001194 conn(0x7f5ac40b3d20 msgr2=0x7f5ac40b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:40.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/57001194 shutdown_connections 2024-04-22T05:50:40.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/57001194 wait complete. 2024-04-22T05:50:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 Processor -- start 2024-04-22T05:50:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 -- start start 2024-04-22T05:50:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5ac41429f0 0x7f5ac40aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac40ab390 0x7f5ac41d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac41d70f0 0x7f5ac4148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac40acc20 con 0x7f5ac41d70f0 2024-04-22T05:50:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac40a9ea0 con 0x7f5ac40ab390 2024-04-22T05:50:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac40ace20 con 0x7f5ac41429f0 2024-04-22T05:50:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac37fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5ac41429f0 0x7f5ac40aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac37fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5ac41429f0 0x7f5ac40aaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:34886/0 (socket says 172.21.15.40:34886) 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac37fe640 1 -- 172.21.15.40:0/2413359055 learned_addr learned my addr 172.21.15.40:0/2413359055 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac3fff640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac41d70f0 0x7f5ac4148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac2ffd640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac40ab390 0x7f5ac41d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac37fe640 1 -- 172.21.15.40:0/2413359055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac40ab390 msgr2=0x7f5ac41d6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac37fe640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac40ab390 0x7f5ac41d6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac37fe640 1 -- 172.21.15.40:0/2413359055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac41d70f0 msgr2=0x7f5ac4148f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac37fe640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac41d70f0 0x7f5ac4148f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac37fe640 1 -- 172.21.15.40:0/2413359055 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab4009420 con 0x7f5ac41429f0 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac3fff640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac41d70f0 0x7f5ac4148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac37fe640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5ac41429f0 0x7f5ac40aaa30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4019fa0 tx=0x7f5ab4019110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5aabfff640 1 -- 172.21.15.40:0/2413359055 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4003b30 con 0x7f5ac41429f0 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5aabfff640 1 -- 172.21.15.40:0/2413359055 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab4003c90 con 0x7f5ac41429f0 2024-04-22T05:50:40.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/2413359055 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac41d7570 con 0x7f5ac41429f0 2024-04-22T05:50:40.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5aabfff640 1 -- 172.21.15.40:0/2413359055 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40ac5b0 con 0x7f5ac41429f0 2024-04-22T05:50:40.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.202+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/2413359055 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac40ab7d0 con 0x7f5ac41429f0 2024-04-22T05:50:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.206+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/2413359055 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac4066d70 con 0x7f5ac41429f0 2024-04-22T05:50:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.206+0000 7f5aabfff640 1 -- 172.21.15.40:0/2413359055 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f5ab40b3040 con 0x7f5ac41429f0 2024-04-22T05:50:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.206+0000 7f5aabfff640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f5ab0066600 0x7f5ab0068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.206+0000 7f5aabfff640 1 -- 172.21.15.40:0/2413359055 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f5ab411dcc0 con 0x7f5ac41429f0 2024-04-22T05:50:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.206+0000 7f5ac2ffd640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f5ab0066600 0x7f5ab0068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:40.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.206+0000 7f5ac2ffd640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f5ab0066600 0x7f5ab0068ac0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005890 tx=0x7f5ab8001db0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:40.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.210+0000 7f5aabfff640 1 -- 172.21.15.40:0/2413359055 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f5ab40eb450 con 0x7f5ac41429f0 2024-04-22T05:50:40.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:40.374+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/2413359055 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5ac40717c0 con 0x7f5ac41429f0 2024-04-22T05:50:41.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.034+0000 7f5aabfff640 1 -- 172.21.15.40:0/2413359055 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7f5ab40b10c0 con 0x7f5ac41429f0 2024-04-22T05:50:41.037 INFO:teuthology.orchestra.run.smithi040.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-22T05:50:41.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.038+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/2413359055 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f5ab0066600 msgr2=0x7f5ab0068ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.038+0000 7f5ac9a6a640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f5ab0066600 0x7f5ab0068ac0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005890 tx=0x7f5ab8001db0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.038+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/2413359055 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5ac41429f0 msgr2=0x7f5ac40aaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.038+0000 7f5ac9a6a640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5ac41429f0 0x7f5ac40aaa30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4019fa0 tx=0x7f5ab4019110 comp rx=0 tx=0).stop 2024-04-22T05:50:41.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.038+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/2413359055 shutdown_connections 2024-04-22T05:50:41.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.038+0000 7f5ac9a6a640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f5ab0066600 0x7f5ab0068ac0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.038+0000 7f5ac9a6a640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac40ab390 0x7f5ac41d6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.038+0000 7f5ac9a6a640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ac41d70f0 0x7f5ac4148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.038+0000 7f5ac9a6a640 1 --2- 172.21.15.40:0/2413359055 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5ac41429f0 0x7f5ac40aaa30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.038+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/2413359055 >> 172.21.15.40:0/2413359055 conn(0x7f5ac40b3d20 msgr2=0x7f5ac4065d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:41.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.038+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/2413359055 shutdown_connections 2024-04-22T05:50:41.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.038+0000 7f5ac9a6a640 1 -- 172.21.15.40:0/2413359055 wait complete. 2024-04-22T05:50:41.064 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-22T05:50:41.064 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-22T05:50:41.064 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-22T05:50:41.064 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-22T05:50:41.065 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-22T05:50:41.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.182+0000 7fc857446640 1 Processor -- start 2024-04-22T05:50:41.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc857446640 1 -- start start 2024-04-22T05:50:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc857446640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8500ad370 0x7fc8501444c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc857446640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc850144a00 0x7fc85014b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc857446640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8500aca00 con 0x7fc8500af660 2024-04-22T05:50:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc857446640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8500acc70 con 0x7fc8500ad370 2024-04-22T05:50:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc857446640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8500acf50 con 0x7fc850144a00 2024-04-22T05:50:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc8549e1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8500ad370 0x7fc8501444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc8549e1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8500ad370 0x7fc8501444c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59386/0 (socket says 172.21.15.40:59386) 2024-04-22T05:50:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc8559e3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc850144a00 0x7fc85014b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc8549e1640 1 -- 172.21.15.40:0/2635678191 learned_addr learned my addr 172.21.15.40:0/2635678191 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc8549e1640 1 -- 172.21.15.40:0/2635678191 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc850144a00 msgr2=0x7fc85014b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc8549e1640 1 --2- 172.21.15.40:0/2635678191 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc850144a00 0x7fc85014b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc8549e1640 1 -- 172.21.15.40:0/2635678191 >> v1:172.21.15.62:6789/0 conn(0x7fc8500af660 legacy=0x7fc850069e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc8549e1640 1 -- 172.21.15.40:0/2635678191 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8500ada90 con 0x7fc8500ad370 2024-04-22T05:50:41.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc8549e1640 1 --2- 172.21.15.40:0/2635678191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8500ad370 0x7fc8501444c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc840009870 tx=0x7fc84001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b07ab591cf0600 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:41.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc847fff640 1 -- 172.21.15.40:0/2635678191 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8400aa430 con 0x7fc8500ad370 2024-04-22T05:50:41.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc847fff640 1 -- 172.21.15.40:0/2635678191 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8400aa590 con 0x7fc8500ad370 2024-04-22T05:50:41.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.186+0000 7fc847fff640 1 -- 172.21.15.40:0/2635678191 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8400aa920 con 0x7fc8500ad370 2024-04-22T05:50:41.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 -- 172.21.15.40:0/2635678191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8500ad370 msgr2=0x7fc8501444c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 --2- 172.21.15.40:0/2635678191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8500ad370 0x7fc8501444c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc840009870 tx=0x7fc84001c510 comp rx=0 tx=0).stop 2024-04-22T05:50:41.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 -- 172.21.15.40:0/2635678191 shutdown_connections 2024-04-22T05:50:41.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 --2- 172.21.15.40:0/2635678191 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc850144a00 0x7fc85014b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 --2- 172.21.15.40:0/2635678191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8500ad370 0x7fc8501444c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 -- 172.21.15.40:0/2635678191 >> 172.21.15.40:0/2635678191 conn(0x7fc85006b7c0 msgr2=0x7fc8500b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:41.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 -- 172.21.15.40:0/2635678191 shutdown_connections 2024-04-22T05:50:41.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 -- 172.21.15.40:0/2635678191 wait complete. 2024-04-22T05:50:41.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 Processor -- start 2024-04-22T05:50:41.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 -- start start 2024-04-22T05:50:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8500ad370 0x7fc85013a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc8500af660 0x7fc8500b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc850144a00 0x7fc85013c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85013c650 con 0x7fc8500ad370 2024-04-22T05:50:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85013c850 con 0x7fc850144a00 2024-04-22T05:50:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc850134be0 con 0x7fc8500af660 2024-04-22T05:50:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc8549e1640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc8500af660 0x7fc8500b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc8551e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8500ad370 0x7fc85013a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc8549e1640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc8500af660 0x7fc8500b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:34896/0 (socket says 172.21.15.40:34896) 2024-04-22T05:50:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc8549e1640 1 -- 172.21.15.40:0/1430512053 learned_addr learned my addr 172.21.15.40:0/1430512053 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc8559e3640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc850144a00 0x7fc85013c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc8549e1640 1 -- 172.21.15.40:0/1430512053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc850144a00 msgr2=0x7fc85013c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc8549e1640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc850144a00 0x7fc85013c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc8549e1640 1 -- 172.21.15.40:0/1430512053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8500ad370 msgr2=0x7fc85013a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc8549e1640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8500ad370 0x7fc85013a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc8549e1640 1 -- 172.21.15.40:0/1430512053 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8400094d0 con 0x7fc8500af660 2024-04-22T05:50:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc8559e3640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc850144a00 0x7fc85013c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T05:50:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc8551e2640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8500ad370 0x7fc85013a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:50:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc8549e1640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc8500af660 0x7fc8500b2a00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc84001abc0 tx=0x7fc84001ad50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc845ffb640 1 -- 172.21.15.40:0/1430512053 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8400aa430 con 0x7fc8500af660 2024-04-22T05:50:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc845ffb640 1 -- 172.21.15.40:0/1430512053 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8400b3820 con 0x7fc8500af660 2024-04-22T05:50:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 -- 172.21.15.40:0/1430512053 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc850134e00 con 0x7fc8500af660 2024-04-22T05:50:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc845ffb640 1 -- 172.21.15.40:0/1430512053 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8400b3bd0 con 0x7fc8500af660 2024-04-22T05:50:41.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.190+0000 7fc857446640 1 -- 172.21.15.40:0/1430512053 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8501352d0 con 0x7fc8500af660 2024-04-22T05:50:41.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.194+0000 7fc857446640 1 -- 172.21.15.40:0/1430512053 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8501354c0 con 0x7fc8500af660 2024-04-22T05:50:41.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.194+0000 7fc845ffb640 1 -- 172.21.15.40:0/1430512053 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7fc8400b3d30 con 0x7fc8500af660 2024-04-22T05:50:41.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.194+0000 7fc845ffb640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fc834066600 0x7fc834068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.194+0000 7fc845ffb640 1 -- 172.21.15.40:0/1430512053 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc84011e720 con 0x7fc8500af660 2024-04-22T05:50:41.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.194+0000 7fc8551e2640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fc834066600 0x7fc834068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.194+0000 7fc8551e2640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fc834066600 0x7fc834068ac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc84c007fd0 tx=0x7fc84c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.198+0000 7fc845ffb640 1 -- 172.21.15.40:0/1430512053 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7fc8400ebea0 con 0x7fc8500af660 2024-04-22T05:50:41.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.398+0000 7fc857446640 1 -- 172.21.15.40:0/1430512053 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc850066d70 con 0x7fc8500af660 2024-04-22T05:50:41.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.402+0000 7fc845ffb640 1 -- 172.21.15.40:0/1430512053 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+164790 (secure 0 0 0) 0x7fc8400ef4c0 con 0x7fc8500af660 2024-04-22T05:50:41.405 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:50:41.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.406+0000 7fc857446640 1 -- 172.21.15.40:0/1430512053 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fc834066600 msgr2=0x7fc834068ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.406+0000 7fc857446640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fc834066600 0x7fc834068ac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc84c007fd0 tx=0x7fc84c009040 comp rx=0 tx=0).stop 2024-04-22T05:50:41.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.406+0000 7fc857446640 1 -- 172.21.15.40:0/1430512053 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc8500af660 msgr2=0x7fc8500b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.406+0000 7fc857446640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc8500af660 0x7fc8500b2a00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc84001abc0 tx=0x7fc84001ad50 comp rx=0 tx=0).stop 2024-04-22T05:50:41.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.406+0000 7fc857446640 1 -- 172.21.15.40:0/1430512053 shutdown_connections 2024-04-22T05:50:41.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.406+0000 7fc857446640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fc834066600 0x7fc834068ac0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.406+0000 7fc857446640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc850144a00 0x7fc85013c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.406+0000 7fc857446640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc8500af660 0x7fc8500b2a00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.406+0000 7fc857446640 1 --2- 172.21.15.40:0/1430512053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8500ad370 0x7fc85013a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.406+0000 7fc857446640 1 -- 172.21.15.40:0/1430512053 >> 172.21.15.40:0/1430512053 conn(0x7fc85006b7c0 msgr2=0x7fc8500707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:41.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.406+0000 7fc857446640 1 -- 172.21.15.40:0/1430512053 shutdown_connections 2024-04-22T05:50:41.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.406+0000 7fc857446640 1 -- 172.21.15.40:0/1430512053 wait complete. 2024-04-22T05:50:41.424 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":4,"flags":0,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":24860},{"type":"v1","addr":"172.21.15.62:6833","nonce":24860}]},"active_addr":"172.21.15.62:6833/24860","active_change":"2024-04-22T05:50:32.925880+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4098,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":2442443451}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":56812651}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":1591942167}]}]} 2024-04-22T05:50:41.430 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-22T05:50:41.430 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-22T05:50:41.430 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T05:50:41.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f8447e09640 1 Processor -- start 2024-04-22T05:50:41.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f8447e09640 1 -- start start 2024-04-22T05:50:41.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f8447e09640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f84400afa20 0x7f8440144560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f8447e09640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8440144aa0 0x7f8440070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f8447e09640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f844006c880 0x7f84400714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f8447e09640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8440071a20 con 0x7f8440144aa0 2024-04-22T05:50:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f8447e09640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f844013ecb0 con 0x7f844006c880 2024-04-22T05:50:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f8447e09640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844006b190 con 0x7f84400afa20 2024-04-22T05:50:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f84453a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8440144aa0 0x7f8440070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f84453a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8440144aa0 0x7f8440070fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59410/0 (socket says 172.21.15.40:59410) 2024-04-22T05:50:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f84453a4640 1 -- 172.21.15.40:0/3742538662 learned_addr learned my addr 172.21.15.40:0/3742538662 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f8445ba5640 1 --2- 172.21.15.40:0/3742538662 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f84400afa20 0x7f8440144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f84463a6640 1 --2- 172.21.15.40:0/3742538662 >> v2:172.21.15.62:3300/0 conn(0x7f844006c880 0x7f84400714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f84453a4640 1 -- 172.21.15.40:0/3742538662 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f84400afa20 msgr2=0x7f8440144560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f84453a4640 1 --2- 172.21.15.40:0/3742538662 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f84400afa20 0x7f8440144560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f84453a4640 1 -- 172.21.15.40:0/3742538662 >> v2:172.21.15.62:3300/0 conn(0x7f844006c880 msgr2=0x7f84400714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f84453a4640 1 --2- 172.21.15.40:0/3742538662 >> v2:172.21.15.62:3300/0 conn(0x7f844006c880 0x7f84400714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f84453a4640 1 -- 172.21.15.40:0/3742538662 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844006b400 con 0x7f8440144aa0 2024-04-22T05:50:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f84463a6640 1 --2- 172.21.15.40:0/3742538662 >> v2:172.21.15.62:3300/0 conn(0x7f844006c880 0x7f84400714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T05:50:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.546+0000 7f84453a4640 1 --2- 172.21.15.40:0/3742538662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8440144aa0 0x7f8440070fa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f844006d070 tx=0x7f843001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33506e6afdc163dd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8444ba3640 1 -- 172.21.15.40:0/3742538662 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84300ab600 con 0x7f8440144aa0 2024-04-22T05:50:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8444ba3640 1 -- 172.21.15.40:0/3742538662 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84300a7050 con 0x7f8440144aa0 2024-04-22T05:50:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8444ba3640 1 -- 172.21.15.40:0/3742538662 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84300ab980 con 0x7f8440144aa0 2024-04-22T05:50:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 -- 172.21.15.40:0/3742538662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8440144aa0 msgr2=0x7f8440070fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 --2- 172.21.15.40:0/3742538662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8440144aa0 0x7f8440070fa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f844006d070 tx=0x7f843001c3e0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 -- 172.21.15.40:0/3742538662 shutdown_connections 2024-04-22T05:50:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 --2- 172.21.15.40:0/3742538662 >> v2:172.21.15.62:3300/0 conn(0x7f844006c880 0x7f84400714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 --2- 172.21.15.40:0/3742538662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8440144aa0 0x7f8440070fa0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 --2- 172.21.15.40:0/3742538662 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f84400afa20 0x7f8440144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 -- 172.21.15.40:0/3742538662 >> 172.21.15.40:0/3742538662 conn(0x7f84400b4cc0 msgr2=0x7f84400b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:41.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 -- 172.21.15.40:0/3742538662 shutdown_connections 2024-04-22T05:50:41.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 -- 172.21.15.40:0/3742538662 wait complete. 2024-04-22T05:50:41.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 Processor -- start 2024-04-22T05:50:41.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 -- start start 2024-04-22T05:50:41.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844006c880 0x7f844013d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f84400afa20 0x7f8440144210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8440144aa0 0x7f84401d9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8440144750 con 0x7f844006c880 2024-04-22T05:50:41.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844013e2b0 con 0x7f8440144aa0 2024-04-22T05:50:41.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84400ac170 con 0x7f84400afa20 2024-04-22T05:50:41.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8445ba5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844006c880 0x7f844013d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f84453a4640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f84400afa20 0x7f8440144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8445ba5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844006c880 0x7f844013d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59426/0 (socket says 172.21.15.40:59426) 2024-04-22T05:50:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8445ba5640 1 -- 172.21.15.40:0/512478303 learned_addr learned my addr 172.21.15.40:0/512478303 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f84463a6640 1 --2- 172.21.15.40:0/512478303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8440144aa0 0x7f84401d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8445ba5640 1 -- 172.21.15.40:0/512478303 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f84400afa20 msgr2=0x7f8440144210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8445ba5640 1 --2- 172.21.15.40:0/512478303 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f84400afa20 0x7f8440144210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8445ba5640 1 -- 172.21.15.40:0/512478303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8440144aa0 msgr2=0x7f84401d9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8445ba5640 1 --2- 172.21.15.40:0/512478303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8440144aa0 0x7f84401d9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8445ba5640 1 -- 172.21.15.40:0/512478303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84300094d0 con 0x7f844006c880 2024-04-22T05:50:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8445ba5640 1 --2- 172.21.15.40:0/512478303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844006c880 0x7f844013d8c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f843c00e930 tx=0x7f843c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f84367fc640 1 -- 172.21.15.40:0/512478303 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c009690 con 0x7f844006c880 2024-04-22T05:50:41.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f84367fc640 1 -- 172.21.15.40:0/512478303 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843c00ca00 con 0x7f844006c880 2024-04-22T05:50:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 -- 172.21.15.40:0/512478303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84400ab860 con 0x7f844006c880 2024-04-22T05:50:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f84367fc640 1 -- 172.21.15.40:0/512478303 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c00cd70 con 0x7f844006c880 2024-04-22T05:50:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.550+0000 7f8447e09640 1 -- 172.21.15.40:0/512478303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84401d9720 con 0x7f844006c880 2024-04-22T05:50:41.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.554+0000 7f8447e09640 1 -- 172.21.15.40:0/512478303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84400abdb0 con 0x7f844006c880 2024-04-22T05:50:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.554+0000 7f84367fc640 1 -- 172.21.15.40:0/512478303 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f843c013470 con 0x7f844006c880 2024-04-22T05:50:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.554+0000 7f84367fc640 1 --2- 172.21.15.40:0/512478303 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8424066600 0x7f8424068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.554+0000 7f84367fc640 1 -- 172.21.15.40:0/512478303 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f843c083f50 con 0x7f844006c880 2024-04-22T05:50:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.554+0000 7f84453a4640 1 --2- 172.21.15.40:0/512478303 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8424066600 0x7f8424068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.554+0000 7f84453a4640 1 --2- 172.21.15.40:0/512478303 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8424066600 0x7f8424068ac0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f84300ad000 tx=0x7f84300a6000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:41.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.558+0000 7f84367fc640 1 -- 172.21.15.40:0/512478303 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f843c051790 con 0x7f844006c880 2024-04-22T05:50:41.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.722+0000 7f8447e09640 1 -- 172.21.15.40:0/512478303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8440066d70 con 0x7f844006c880 2024-04-22T05:50:41.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.722+0000 7f84367fc640 1 -- 172.21.15.40:0/512478303 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+14133 (secure 0 0 0) 0x7f843c017090 con 0x7f844006c880 2024-04-22T05:50:41.725 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:50:41.725 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"fsid":"36e9f223-2858-4559-8f74-0be5d30b8b0d","created":"2024-04-22T05:50:28.068495+0000","modified":"2024-04-22T05:50:41.023845+0000","last_up_change":"2024-04-22T05:50:37.978413+0000","last_in_change":"2024-04-22T05:50:30.434344+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T05:50:39.010995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-22T05:50:39.240779+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1997ac14-110c-4d7e-8ac5-d0ac4f70af75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":24983},{"type":"v1","addr":"172.21.15.40:6818","nonce":24983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":24983},{"type":"v1","addr":"172.21.15.40:6822","nonce":24983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":24983},{"type":"v1","addr":"172.21.15.40:6830","nonce":24983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":24983},{"type":"v1","addr":"172.21.15.40:6826","nonce":24983}]},"public_addr":"172.21.15.40:6818/24983","cluster_addr":"172.21.15.40:6822/24983","heartbeat_back_addr":"172.21.15.40:6830/24983","heartbeat_front_addr":"172.21.15.40:6826/24983","state":["exists","up"]},{"osd":1,"uuid":"cb6b2995-f7ce-4a68-ac16-d1690cc2b9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":24984},{"type":"v1","addr":"172.21.15.40:6801","nonce":24984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":24984},{"type":"v1","addr":"172.21.15.40:6803","nonce":24984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":24984},{"type":"v1","addr":"172.21.15.40:6807","nonce":24984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":24984},{"type":"v1","addr":"172.21.15.40:6805","nonce":24984}]},"public_addr":"172.21.15.40:6801/24984","cluster_addr":"172.21.15.40:6803/24984","heartbeat_back_addr":"172.21.15.40:6807/24984","heartbeat_front_addr":"172.21.15.40:6805/24984","state":["exists","up"]},{"osd":2,"uuid":"a5634ce0-a08d-403f-bb05-eadb21417851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":24986},{"type":"v1","addr":"172.21.15.40:6809","nonce":24986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":24986},{"type":"v1","addr":"172.21.15.40:6811","nonce":24986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":24986},{"type":"v1","addr":"172.21.15.40:6815","nonce":24986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":24986},{"type":"v1","addr":"172.21.15.40:6813","nonce":24986}]},"public_addr":"172.21.15.40:6809/24986","cluster_addr":"172.21.15.40:6811/24986","heartbeat_back_addr":"172.21.15.40:6815/24986","heartbeat_front_addr":"172.21.15.40:6813/24986","state":["exists","up"]},{"osd":3,"uuid":"bada948d-c18e-4bf2-8902-9f383fcd1b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":24985},{"type":"v1","addr":"172.21.15.40:6819","nonce":24985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":24985},{"type":"v1","addr":"172.21.15.40:6823","nonce":24985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":24985},{"type":"v1","addr":"172.21.15.40:6831","nonce":24985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":24985},{"type":"v1","addr":"172.21.15.40:6827","nonce":24985}]},"public_addr":"172.21.15.40:6819/24985","cluster_addr":"172.21.15.40:6823/24985","heartbeat_back_addr":"172.21.15.40:6831/24985","heartbeat_front_addr":"172.21.15.40:6827/24985","state":["exists","up"]},{"osd":4,"uuid":"4ad76cfa-9e85-421e-bd2e-61006dd694bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":25135},{"type":"v1","addr":"172.21.15.62:6817","nonce":25135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":25135},{"type":"v1","addr":"172.21.15.62:6819","nonce":25135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":25135},{"type":"v1","addr":"172.21.15.62:6823","nonce":25135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":25135},{"type":"v1","addr":"172.21.15.62:6821","nonce":25135}]},"public_addr":"172.21.15.62:6817/25135","cluster_addr":"172.21.15.62:6819/25135","heartbeat_back_addr":"172.21.15.62:6823/25135","heartbeat_front_addr":"172.21.15.62:6821/25135","state":["exists","up"]},{"osd":5,"uuid":"b3b6f5a2-d949-429c-bfc4-b6a7333d8184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":25137},{"type":"v1","addr":"172.21.15.62:6825","nonce":25137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":25137},{"type":"v1","addr":"172.21.15.62:6827","nonce":25137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":25137},{"type":"v1","addr":"172.21.15.62:6831","nonce":25137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":25137},{"type":"v1","addr":"172.21.15.62:6829","nonce":25137}]},"public_addr":"172.21.15.62:6825/25137","cluster_addr":"172.21.15.62:6827/25137","heartbeat_back_addr":"172.21.15.62:6831/25137","heartbeat_front_addr":"172.21.15.62:6829/25137","state":["exists","up"]},{"osd":6,"uuid":"00c5caf5-710a-4162-93fa-917a4823ffaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":25136},{"type":"v1","addr":"172.21.15.62:6801","nonce":25136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":25136},{"type":"v1","addr":"172.21.15.62:6803","nonce":25136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":25136},{"type":"v1","addr":"172.21.15.62:6807","nonce":25136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":25136},{"type":"v1","addr":"172.21.15.62:6805","nonce":25136}]},"public_addr":"172.21.15.62:6801/25136","cluster_addr":"172.21.15.62:6803/25136","heartbeat_back_addr":"172.21.15.62:6807/25136","heartbeat_front_addr":"172.21.15.62:6805/25136","state":["exists","up"]},{"osd":7,"uuid":"d17323e1-9cec-4bcb-b603-7f9fda35b5f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":25138},{"type":"v1","addr":"172.21.15.62:6809","nonce":25138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":25138},{"type":"v1","addr":"172.21.15.62:6811","nonce":25138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":25138},{"type":"v1","addr":"172.21.15.62:6815","nonce":25138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":25138},{"type":"v1","addr":"172.21.15.62:6813","nonce":25138}]},"public_addr":"172.21.15.62:6809/25138","cluster_addr":"172.21.15.62:6811/25138","heartbeat_back_addr":"172.21.15.62:6815/25138","heartbeat_front_addr":"172.21.15.62:6813/25138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.967711+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.944884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.961822+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.419674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.997808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:36.052542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.948229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:36.683341+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T05:50:41.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.726+0000 7f8447e09640 1 -- 172.21.15.40:0/512478303 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8424066600 msgr2=0x7f8424068ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.726+0000 7f8447e09640 1 --2- 172.21.15.40:0/512478303 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8424066600 0x7f8424068ac0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f84300ad000 tx=0x7f84300a6000 comp rx=0 tx=0).stop 2024-04-22T05:50:41.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.726+0000 7f8447e09640 1 -- 172.21.15.40:0/512478303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844006c880 msgr2=0x7f844013d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.726+0000 7f8447e09640 1 --2- 172.21.15.40:0/512478303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844006c880 0x7f844013d8c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f843c00e930 tx=0x7f843c00edf0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.726+0000 7f8447e09640 1 -- 172.21.15.40:0/512478303 shutdown_connections 2024-04-22T05:50:41.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.726+0000 7f8447e09640 1 --2- 172.21.15.40:0/512478303 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8424066600 0x7f8424068ac0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.726+0000 7f8447e09640 1 --2- 172.21.15.40:0/512478303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8440144aa0 0x7f84401d9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.726+0000 7f8447e09640 1 --2- 172.21.15.40:0/512478303 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f84400afa20 0x7f8440144210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.726+0000 7f8447e09640 1 --2- 172.21.15.40:0/512478303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f844006c880 0x7f844013d8c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.726+0000 7f8447e09640 1 -- 172.21.15.40:0/512478303 >> 172.21.15.40:0/512478303 conn(0x7f84400b4cc0 msgr2=0x7f84400b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:41.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.726+0000 7f8447e09640 1 -- 172.21.15.40:0/512478303 shutdown_connections 2024-04-22T05:50:41.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.726+0000 7f8447e09640 1 -- 172.21.15.40:0/512478303 wait complete. 2024-04-22T05:50:41.745 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-22T05:50:41.746 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T05:50:41.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bfd7cf640 1 Processor -- start 2024-04-22T05:50:41.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bfd7cf640 1 -- start start 2024-04-22T05:50:41.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bfd7cf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9bf81424b0 0x7f9bf8142de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bfd7cf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf80706c0 con 0x7f9bf813bac0 2024-04-22T05:50:41.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bfd7cf640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf8142890 con 0x7f9bf80ae9e0 2024-04-22T05:50:41.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bfd7cf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf80ad530 con 0x7f9bf81424b0 2024-04-22T05:50:41.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bf6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9bf81424b0 0x7f9bf8142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bf6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9bf81424b0 0x7f9bf8142de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59436/0 (socket says 172.21.15.40:59436) 2024-04-22T05:50:41.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bf6ffd640 1 -- 172.21.15.40:0/1843954200 learned_addr learned my addr 172.21.15.40:0/1843954200 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:41.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bf77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9bf813bac0 0x7f9bf813bf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56004/0 (socket says 172.21.15.40:56004) 2024-04-22T05:50:41.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bf6ffd640 1 -- 172.21.15.40:0/1843954200 >> v1:172.21.15.62:6789/0 conn(0x7f9bf80ae9e0 legacy=0x7f9bf806ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bf6ffd640 1 -- 172.21.15.40:0/1843954200 >> v1:172.21.15.40:6789/0 conn(0x7f9bf813bac0 legacy=0x7f9bf813bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bf6ffd640 1 -- 172.21.15.40:0/1843954200 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80addb0 con 0x7f9bf81424b0 2024-04-22T05:50:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.870+0000 7f9bf5ffb640 1 -- 172.21.15.40:0/1843954200 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202475215 0 0) 0x7f9bf80706c0 con 0x7f9bf813bac0 2024-04-22T05:50:41.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.874+0000 7f9bf6ffd640 1 --2- 172.21.15.40:0/1843954200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf81424b0 0x7f9bf8142de0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009870 tx=0x7f9be801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398b2d755a7f02ca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:41.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.874+0000 7f9bf5ffb640 1 -- 172.21.15.40:0/1843954200 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80ad660 con 0x7f9bf81424b0 2024-04-22T05:50:41.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.874+0000 7f9bf5ffb640 1 -- 172.21.15.40:0/1843954200 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be80ad7c0 con 0x7f9bf81424b0 2024-04-22T05:50:41.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.874+0000 7f9bf5ffb640 1 -- 172.21.15.40:0/1843954200 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80adb50 con 0x7f9bf81424b0 2024-04-22T05:50:41.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.874+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/1843954200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf81424b0 msgr2=0x7f9bf8142de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.874+0000 7f9bfd7cf640 1 --2- 172.21.15.40:0/1843954200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf81424b0 0x7f9bf8142de0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009870 tx=0x7f9be801c420 comp rx=0 tx=0).stop 2024-04-22T05:50:41.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.874+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/1843954200 shutdown_connections 2024-04-22T05:50:41.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.874+0000 7f9bfd7cf640 1 --2- 172.21.15.40:0/1843954200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf81424b0 0x7f9bf8142de0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.874+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/1843954200 >> 172.21.15.40:0/1843954200 conn(0x7f9bf80b28a0 msgr2=0x7f9bf80b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:41.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.874+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/1843954200 shutdown_connections 2024-04-22T05:50:41.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.874+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/1843954200 wait complete. 2024-04-22T05:50:41.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bfd7cf640 1 Processor -- start 2024-04-22T05:50:41.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bfd7cf640 1 -- start start 2024-04-22T05:50:41.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bfd7cf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf80ae9e0 0x7f9bf80a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bfd7cf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf813bac0 0x7f9bf80aa160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bfd7cf640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bf81424b0 0x7f9bf8131940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bfd7cf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf8137f90 con 0x7f9bf80ae9e0 2024-04-22T05:50:41.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bfd7cf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80aa6a0 con 0x7f9bf813bac0 2024-04-22T05:50:41.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bfd7cf640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80abf20 con 0x7f9bf81424b0 2024-04-22T05:50:41.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bf6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf80ae9e0 0x7f9bf80a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bf77fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bf81424b0 0x7f9bf8131940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bf6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf80ae9e0 0x7f9bf80a9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59438/0 (socket says 172.21.15.40:59438) 2024-04-22T05:50:41.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bf6ffd640 1 -- 172.21.15.40:0/246203596 learned_addr learned my addr 172.21.15.40:0/246203596 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:41.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bf67fc640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf813bac0 0x7f9bf80aa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bf6ffd640 1 -- 172.21.15.40:0/246203596 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bf81424b0 msgr2=0x7f9bf8131940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bf6ffd640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bf81424b0 0x7f9bf8131940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bf6ffd640 1 -- 172.21.15.40:0/246203596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf813bac0 msgr2=0x7f9bf80aa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:41.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bf6ffd640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf813bac0 0x7f9bf80aa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:41.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bf6ffd640 1 -- 172.21.15.40:0/246203596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be80094d0 con 0x7f9bf80ae9e0 2024-04-22T05:50:41.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bf77fe640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bf81424b0 0x7f9bf8131940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:50:41.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bf6ffd640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf80ae9e0 0x7f9bf80a9c20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9be80b0040 tx=0x7f9be801ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:41.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bdf7fe640 1 -- 172.21.15.40:0/246203596 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80b88f0 con 0x7f9bf80ae9e0 2024-04-22T05:50:41.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bdf7fe640 1 -- 172.21.15.40:0/246203596 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be80b8a70 con 0x7f9bf80ae9e0 2024-04-22T05:50:41.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/246203596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf80ac140 con 0x7f9bf80ae9e0 2024-04-22T05:50:41.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bdf7fe640 1 -- 172.21.15.40:0/246203596 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80b8e20 con 0x7f9bf80ae9e0 2024-04-22T05:50:41.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/246203596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf8132050 con 0x7f9bf80ae9e0 2024-04-22T05:50:41.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/246203596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf80abc60 con 0x7f9bf80ae9e0 2024-04-22T05:50:41.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.878+0000 7f9bdf7fe640 1 -- 172.21.15.40:0/246203596 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f9be80b4040 con 0x7f9bf80ae9e0 2024-04-22T05:50:41.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.882+0000 7f9bdf7fe640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9be4066650 0x7f9be4068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:41.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.882+0000 7f9bdf7fe640 1 -- 172.21.15.40:0/246203596 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9be811e610 con 0x7f9bf80ae9e0 2024-04-22T05:50:41.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.882+0000 7f9bf67fc640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9be4066650 0x7f9be4068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:41.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.882+0000 7f9bf67fc640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9be4066650 0x7f9be4068b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9bf81db400 tx=0x7f9bec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:41.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.882+0000 7f9bdf7fe640 1 -- 172.21.15.40:0/246203596 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9be80ec240 con 0x7f9bf80ae9e0 2024-04-22T05:50:41.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:41.890+0000 7f9bdf7fe640 1 -- 172.21.15.40:0/246203596 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f9be80c3110 con 0x7f9bf80ae9e0 2024-04-22T05:50:42.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.050+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/246203596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bf80abc60 con 0x7f9bf80ae9e0 2024-04-22T05:50:42.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.054+0000 7f9bdf7fe640 1 -- 172.21.15.40:0/246203596 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14120 (secure 0 0 0) 0x7f9bf80abc60 con 0x7f9bf80ae9e0 2024-04-22T05:50:42.055 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:50:42.055 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":18,"fsid":"36e9f223-2858-4559-8f74-0be5d30b8b0d","created":"2024-04-22T05:50:28.068495+0000","modified":"2024-04-22T05:50:42.043490+0000","last_up_change":"2024-04-22T05:50:37.978413+0000","last_in_change":"2024-04-22T05:50:30.434344+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T05:50:39.010995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-22T05:50:39.240779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1997ac14-110c-4d7e-8ac5-d0ac4f70af75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":24983},{"type":"v1","addr":"172.21.15.40:6818","nonce":24983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":24983},{"type":"v1","addr":"172.21.15.40:6822","nonce":24983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":24983},{"type":"v1","addr":"172.21.15.40:6830","nonce":24983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":24983},{"type":"v1","addr":"172.21.15.40:6826","nonce":24983}]},"public_addr":"172.21.15.40:6818/24983","cluster_addr":"172.21.15.40:6822/24983","heartbeat_back_addr":"172.21.15.40:6830/24983","heartbeat_front_addr":"172.21.15.40:6826/24983","state":["exists","up"]},{"osd":1,"uuid":"cb6b2995-f7ce-4a68-ac16-d1690cc2b9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":24984},{"type":"v1","addr":"172.21.15.40:6801","nonce":24984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":24984},{"type":"v1","addr":"172.21.15.40:6803","nonce":24984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":24984},{"type":"v1","addr":"172.21.15.40:6807","nonce":24984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":24984},{"type":"v1","addr":"172.21.15.40:6805","nonce":24984}]},"public_addr":"172.21.15.40:6801/24984","cluster_addr":"172.21.15.40:6803/24984","heartbeat_back_addr":"172.21.15.40:6807/24984","heartbeat_front_addr":"172.21.15.40:6805/24984","state":["exists","up"]},{"osd":2,"uuid":"a5634ce0-a08d-403f-bb05-eadb21417851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":24986},{"type":"v1","addr":"172.21.15.40:6809","nonce":24986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":24986},{"type":"v1","addr":"172.21.15.40:6811","nonce":24986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":24986},{"type":"v1","addr":"172.21.15.40:6815","nonce":24986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":24986},{"type":"v1","addr":"172.21.15.40:6813","nonce":24986}]},"public_addr":"172.21.15.40:6809/24986","cluster_addr":"172.21.15.40:6811/24986","heartbeat_back_addr":"172.21.15.40:6815/24986","heartbeat_front_addr":"172.21.15.40:6813/24986","state":["exists","up"]},{"osd":3,"uuid":"bada948d-c18e-4bf2-8902-9f383fcd1b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":24985},{"type":"v1","addr":"172.21.15.40:6819","nonce":24985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":24985},{"type":"v1","addr":"172.21.15.40:6823","nonce":24985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":24985},{"type":"v1","addr":"172.21.15.40:6831","nonce":24985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":24985},{"type":"v1","addr":"172.21.15.40:6827","nonce":24985}]},"public_addr":"172.21.15.40:6819/24985","cluster_addr":"172.21.15.40:6823/24985","heartbeat_back_addr":"172.21.15.40:6831/24985","heartbeat_front_addr":"172.21.15.40:6827/24985","state":["exists","up"]},{"osd":4,"uuid":"4ad76cfa-9e85-421e-bd2e-61006dd694bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":25135},{"type":"v1","addr":"172.21.15.62:6817","nonce":25135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":25135},{"type":"v1","addr":"172.21.15.62:6819","nonce":25135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":25135},{"type":"v1","addr":"172.21.15.62:6823","nonce":25135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":25135},{"type":"v1","addr":"172.21.15.62:6821","nonce":25135}]},"public_addr":"172.21.15.62:6817/25135","cluster_addr":"172.21.15.62:6819/25135","heartbeat_back_addr":"172.21.15.62:6823/25135","heartbeat_front_addr":"172.21.15.62:6821/25135","state":["exists","up"]},{"osd":5,"uuid":"b3b6f5a2-d949-429c-bfc4-b6a7333d8184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":25137},{"type":"v1","addr":"172.21.15.62:6825","nonce":25137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":25137},{"type":"v1","addr":"172.21.15.62:6827","nonce":25137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":25137},{"type":"v1","addr":"172.21.15.62:6831","nonce":25137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":25137},{"type":"v1","addr":"172.21.15.62:6829","nonce":25137}]},"public_addr":"172.21.15.62:6825/25137","cluster_addr":"172.21.15.62:6827/25137","heartbeat_back_addr":"172.21.15.62:6831/25137","heartbeat_front_addr":"172.21.15.62:6829/25137","state":["exists","up"]},{"osd":6,"uuid":"00c5caf5-710a-4162-93fa-917a4823ffaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":25136},{"type":"v1","addr":"172.21.15.62:6801","nonce":25136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":25136},{"type":"v1","addr":"172.21.15.62:6803","nonce":25136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":25136},{"type":"v1","addr":"172.21.15.62:6807","nonce":25136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":25136},{"type":"v1","addr":"172.21.15.62:6805","nonce":25136}]},"public_addr":"172.21.15.62:6801/25136","cluster_addr":"172.21.15.62:6803/25136","heartbeat_back_addr":"172.21.15.62:6807/25136","heartbeat_front_addr":"172.21.15.62:6805/25136","state":["exists","up"]},{"osd":7,"uuid":"d17323e1-9cec-4bcb-b603-7f9fda35b5f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":25138},{"type":"v1","addr":"172.21.15.62:6809","nonce":25138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":25138},{"type":"v1","addr":"172.21.15.62:6811","nonce":25138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":25138},{"type":"v1","addr":"172.21.15.62:6815","nonce":25138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":25138},{"type":"v1","addr":"172.21.15.62:6813","nonce":25138}]},"public_addr":"172.21.15.62:6809/25138","cluster_addr":"172.21.15.62:6811/25138","heartbeat_back_addr":"172.21.15.62:6815/25138","heartbeat_front_addr":"172.21.15.62:6813/25138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.967711+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.944884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.961822+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.419674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.997808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:36.052542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.948229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:36.683341+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T05:50:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.058+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/246203596 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9be4066650 msgr2=0x7f9be4068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.058+0000 7f9bfd7cf640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9be4066650 0x7f9be4068b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9bf81db400 tx=0x7f9bec009040 comp rx=0 tx=0).stop 2024-04-22T05:50:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.058+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/246203596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf80ae9e0 msgr2=0x7f9bf80a9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.058+0000 7f9bfd7cf640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf80ae9e0 0x7f9bf80a9c20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9be80b0040 tx=0x7f9be801ac60 comp rx=0 tx=0).stop 2024-04-22T05:50:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.058+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/246203596 shutdown_connections 2024-04-22T05:50:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.058+0000 7f9bfd7cf640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9be4066650 0x7f9be4068b10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.058+0000 7f9bfd7cf640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf813bac0 0x7f9bf80aa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.058+0000 7f9bfd7cf640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bf81424b0 0x7f9bf8131940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.058+0000 7f9bfd7cf640 1 --2- 172.21.15.40:0/246203596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf80ae9e0 0x7f9bf80a9c20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.058+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/246203596 >> 172.21.15.40:0/246203596 conn(0x7f9bf80b28a0 msgr2=0x7f9bf80b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.058+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/246203596 shutdown_connections 2024-04-22T05:50:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.058+0000 7f9bfd7cf640 1 -- 172.21.15.40:0/246203596 wait complete. 2024-04-22T05:50:42.076 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-22T05:50:42.076 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-22T05:50:42.076 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-22T05:50:42.077 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-22T05:50:42.077 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-22T05:50:42.077 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-22T05:50:42.078 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-22T05:50:42.078 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-22T05:50:42.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa27fff640 1 Processor -- start 2024-04-22T05:50:42.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa27fff640 1 -- start start 2024-04-22T05:50:42.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa27fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffa200375e0 0x7ffa200c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa27fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ffa2002b8c0 0x7ffa200c1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa26ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffa200375e0 0x7ffa200c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa26ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffa200375e0 0x7ffa200c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59440/0 (socket says 172.21.15.40:59440) 2024-04-22T05:50:42.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa26ffd640 1 -- 172.21.15.40:0/1334608416 learned_addr learned my addr 172.21.15.40:0/1334608416 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa267fc640 1 --2- 172.21.15.40:0/1334608416 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ffa2002b8c0 0x7ffa200c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa27fff640 1 -- 172.21.15.40:0/1334608416 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa2002b430 con 0x7ffa2002c5b0 2024-04-22T05:50:42.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa27fff640 1 -- 172.21.15.40:0/1334608416 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa2002d5f0 con 0x7ffa200375e0 2024-04-22T05:50:42.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa27fff640 1 -- 172.21.15.40:0/1334608416 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2002d8d0 con 0x7ffa2002b8c0 2024-04-22T05:50:42.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa25ffb640 1 -- 172.21.15.40:0/1334608416 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839626648 0 0) 0x7ffa2002b430 con 0x7ffa2002c5b0 2024-04-22T05:50:42.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa25ffb640 1 -- 172.21.15.40:0/1334608416 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa10003430 con 0x7ffa2002c5b0 2024-04-22T05:50:42.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa26ffd640 1 -- 172.21.15.40:0/1334608416 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ffa2002b8c0 msgr2=0x7ffa200c1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa25ffb640 1 -- 172.21.15.40:0/1334608416 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4258681223 0 0) 0x7ffa10003430 con 0x7ffa2002c5b0 2024-04-22T05:50:42.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa26ffd640 1 --2- 172.21.15.40:0/1334608416 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ffa2002b8c0 0x7ffa200c1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa26ffd640 1 -- 172.21.15.40:0/1334608416 >> v1:172.21.15.40:6789/0 conn(0x7ffa2002c5b0 legacy=0x7ffa200c1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7ffa26ffd640 1 -- 172.21.15.40:0/1334608416 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2002b0c0 con 0x7ffa200375e0 2024-04-22T05:50:42.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7ffa267fc640 1 --2- 172.21.15.40:0/1334608416 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ffa2002b8c0 0x7ffa200c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T05:50:42.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05ccef8640 1 Processor -- start 2024-04-22T05:50:42.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05ccef8640 1 -- start start 2024-04-22T05:50:42.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05ccef8640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f05c80ad400 0x7f05c813a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05ccef8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05c81417f0 0x7f05c8148470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05ccef8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c80ad7e0 con 0x7f05c813af10 2024-04-22T05:50:42.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05ccef8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c8070c30 con 0x7f05c81417f0 2024-04-22T05:50:42.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05ccef8640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c8070e30 con 0x7f05c80ad400 2024-04-22T05:50:42.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7ffa26ffd640 1 --2- 172.21.15.40:0/1334608416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa200375e0 0x7ffa200c7df0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009870 tx=0x7ffa1801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894dfafb97fb9795 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05c7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05c81417f0 0x7f05c8148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05c7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05c81417f0 0x7f05c8148470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59450/0 (socket says 172.21.15.40:59450) 2024-04-22T05:50:42.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05c7fff640 1 -- 172.21.15.40:0/3785108589 learned_addr learned my addr 172.21.15.40:0/3785108589 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05c77fe640 1 --2- 172.21.15.40:0/3785108589 >> v2:172.21.15.62:3300/0 conn(0x7f05c80ad400 0x7f05c813a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7ffa25ffb640 1 -- 172.21.15.40:0/1334608416 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2804e690 con 0x7ffa200375e0 2024-04-22T05:50:42.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.262+0000 7fe2d4d80640 1 Processor -- start 2024-04-22T05:50:42.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.262+0000 7fe2d4d80640 1 -- start start 2024-04-22T05:50:42.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a1689c640 1 Processor -- start 2024-04-22T05:50:42.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a1689c640 1 -- start start 2024-04-22T05:50:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7fe2d4d80640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2d00709e0 0x7fe2d00ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7fe2d4d80640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe2d0070dc0 0x7fe2d01512d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7fe2cf7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2d00709e0 0x7fe2d00ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7fe2cf7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2d00709e0 0x7fe2d00ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:34942/0 (socket says 172.21.15.40:34942) 2024-04-22T05:50:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7fe2ceffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe2d0070dc0 0x7fe2d01512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f01de225640 1 Processor -- start 2024-04-22T05:50:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7fe2ceffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe2d0070dc0 0x7fe2d01512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:58728/0 (socket says 172.21.15.40:58728) 2024-04-22T05:50:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7fe2cf7fe640 1 -- 172.21.15.40:0/1611800090 learned_addr learned my addr 172.21.15.40:0/1611800090 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7fe2d4d80640 1 -- 172.21.15.40:0/1611800090 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d006fca0 con 0x7fe2d0071200 2024-04-22T05:50:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7fe2d4d80640 1 -- 172.21.15.40:0/1611800090 --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d006f190 con 0x7fe2d0070dc0 2024-04-22T05:50:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7fe2d4d80640 1 -- 172.21.15.40:0/1611800090 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d006ff80 con 0x7fe2d00709e0 2024-04-22T05:50:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.266+0000 7fe2ce7fc640 1 -- 172.21.15.40:0/1611800090 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766835768 0 0) 0x7fe2d006fca0 con 0x7fe2d0071200 2024-04-22T05:50:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a1689c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2a08032130 0x7f2a080311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a1689c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a08007220 con 0x7f2a08006e40 2024-04-22T05:50:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a1689c640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a08033190 con 0x7f2a0802f5c0 2024-04-22T05:50:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a1689c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a08032f90 con 0x7f2a08032130 2024-04-22T05:50:42.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05c77fe640 1 -- 172.21.15.40:0/3785108589 >> v2:172.21.15.40:3300/0 conn(0x7f05c81417f0 msgr2=0x7f05c8148470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05c77fe640 1 --2- 172.21.15.40:0/3785108589 >> v2:172.21.15.40:3300/0 conn(0x7f05c81417f0 0x7f05c8148470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05c77fe640 1 -- 172.21.15.40:0/3785108589 >> v1:172.21.15.40:6789/0 conn(0x7f05c813af10 legacy=0x7f05c81410e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:42.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f05c77fe640 1 -- 172.21.15.40:0/3785108589 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c8071110 con 0x7f05c80ad400 2024-04-22T05:50:42.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a1589a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2a08006e40 0x7f2a080e8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56012/0 (socket says 172.21.15.40:56012) 2024-04-22T05:50:42.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a1589a640 1 -- 172.21.15.40:0/3227459380 learned_addr learned my addr 172.21.15.40:0/3227459380 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:50:42.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7ffa25ffb640 1 -- 172.21.15.40:0/1334608416 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa180a7050 con 0x7ffa200375e0 2024-04-22T05:50:42.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7ffa25ffb640 1 -- 172.21.15.40:0/1334608416 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa180ab600 con 0x7ffa200375e0 2024-04-22T05:50:42.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7fe2ce7fc640 1 -- 172.21.15.40:0/1611800090 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2b8003510 con 0x7fe2d0071200 2024-04-22T05:50:42.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7ffa27fff640 1 -- 172.21.15.40:0/1334608416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa200375e0 msgr2=0x7ffa200c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7ffa27fff640 1 --2- 172.21.15.40:0/1334608416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa200375e0 0x7ffa200c7df0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009870 tx=0x7ffa1801c3f0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a14898640 1 -- 172.21.15.40:0/3227459380 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575841435 0 0) 0x7f2a08007220 con 0x7f2a08006e40 2024-04-22T05:50:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a14898640 1 -- 172.21.15.40:0/3227459380 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29fc003530 con 0x7f2a08006e40 2024-04-22T05:50:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a1609b640 1 -- 172.21.15.40:0/3227459380 >> v1:172.21.15.62:6789/0 conn(0x7f2a0802f5c0 legacy=0x7f2a080e29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-04-22T05:50:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a1609b640 1 -- 172.21.15.40:0/3227459380 >> v1:172.21.15.62:6789/0 conn(0x7f2a0802f5c0 legacy=0x7f2a080e29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T05:50:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a1609b640 1 --1- 172.21.15.40:0/3227459380 >> v1:172.21.15.62:6789/0 conn(0x7f2a0802f5c0 0x7f2a080e29a0 :-1 s=OPENED pgs=70 cs=1 l=1).handle_message read tag failed 2024-04-22T05:50:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a1609b640 1 --1- 172.21.15.40:0/3227459380 >> v1:172.21.15.62:6789/0 conn(0x7f2a0802f5c0 0x7f2a080e29a0 :-1 s=OPENED pgs=70 cs=1 l=1).fault on lossy channel, failing 2024-04-22T05:50:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f01de225640 1 -- start start 2024-04-22T05:50:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f01de225640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f01d80aed40 0x7f01d8148730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f01de225640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f01d8148c70 0x7f01d814f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f01de225640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d8073920 con 0x7f01d8073540 2024-04-22T05:50:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f01de225640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d80af560 con 0x7f01d80aed40 2024-04-22T05:50:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f01de225640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8142a90 con 0x7f01d8148c70 2024-04-22T05:50:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a14898640 1 -- 172.21.15.40:0/3227459380 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1489799592 0 0) 0x7f29fc003530 con 0x7f2a08006e40 2024-04-22T05:50:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a14898640 1 -- 172.21.15.40:0/3227459380 >> v2:172.21.15.40:3300/0 conn(0x7f2a08032130 msgr2=0x7f2a080311f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a14898640 1 --2- 172.21.15.40:0/3227459380 >> v2:172.21.15.40:3300/0 conn(0x7f2a08032130 0x7f2a080311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a14898640 1 -- 172.21.15.40:0/3227459380 >> v1:172.21.15.62:6789/0 conn(0x7f2a0802f5c0 legacy=0x7f2a080e29a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-22T05:50:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f01dda24640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f01d8148c70 0x7f01d814f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f01dda24640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f01d8148c70 0x7f01d814f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:34962/0 (socket says 172.21.15.40:34962) 2024-04-22T05:50:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f01dda24640 1 -- 172.21.15.40:0/4076492809 learned_addr learned my addr 172.21.15.40:0/4076492809 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a14898640 1 -- 172.21.15.40:0/3227459380 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a08030cd0 con 0x7f2a08006e40 2024-04-22T05:50:42.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f01dca22640 1 --2- 172.21.15.40:0/4076492809 >> v2:172.21.15.40:3300/0 conn(0x7f01d80aed40 0x7f01d8148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7f2a14898640 1 -- 172.21.15.40:0/3227459380 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268782488 0 0) 0x7f2a0c001680 con 0x7f2a08006e40 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a14898640 1 -- 172.21.15.40:0/3227459380 >> v1:172.21.15.40:6789/0 conn(0x7f2a08006e40 legacy=0x7f2a080e8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a14898640 1 --2- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29fc003980 0x7f29fc003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a14898640 1 --2- 172.21.15.40:0/3227459380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29fc0047d0 0x7f29fc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a14898640 1 --2- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f29fc0052b0 0x7f29fc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a14898640 1 -- 172.21.15.40:0/3227459380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a08032f90 con 0x7f29fc003980 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a14898640 1 -- 172.21.15.40:0/3227459380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc00c200 con 0x7f29fc0047d0 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a14898640 1 -- 172.21.15.40:0/3227459380 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc00c3c0 con 0x7f29fc0052b0 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a1609b640 1 --2- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f29fc0052b0 0x7f29fc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a1589a640 1 --2- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29fc003980 0x7f29fc003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a1589a640 1 -- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29fc003980 msgr2=0x7f29fc003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a1589a640 1 -- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29fc003980 msgr2=0x7f29fc003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a1589a640 1 --2- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29fc003980 0x7f29fc003d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a1589a640 1 --2- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29fc003980 0x7f29fc003d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T05:50:42.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7fe2ceffd640 1 -- 172.21.15.40:0/1611800090 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2d00709e0 msgr2=0x7fe2d00ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7fe2ceffd640 1 --2- 172.21.15.40:0/1611800090 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2d00709e0 0x7fe2d00ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7fe2ceffd640 1 -- 172.21.15.40:0/1611800090 >> v1:172.21.15.40:6789/0 conn(0x7fe2d0071200 legacy=0x7fe2d0151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.270+0000 7fe2ceffd640 1 -- 172.21.15.40:0/1611800090 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d0070260 con 0x7fe2d0070dc0 2024-04-22T05:50:42.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f760aec9640 1 Processor -- start 2024-04-22T05:50:42.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f760aec9640 1 -- start start 2024-04-22T05:50:42.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f05c77fe640 1 --2- 172.21.15.40:0/3785108589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05c80ad400 0x7f05c813a9d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f05c80ada00 tx=0x7f05c001c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=857516ebfe95b8a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f760aec9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76040ae500 0x7f760406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f05c67fc640 1 -- 172.21.15.40:0/3785108589 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c00a58d0 con 0x7f05c80ad400 2024-04-22T05:50:42.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f05c67fc640 1 -- 172.21.15.40:0/3785108589 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c00a5a30 con 0x7f05c80ad400 2024-04-22T05:50:42.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f05c67fc640 1 -- 172.21.15.40:0/3785108589 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c00a5dc0 con 0x7f05c80ad400 2024-04-22T05:50:42.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f760aec9640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f760406e8e0 0x7f76040ad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f760aec9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7604070660 con 0x7f760406e480 2024-04-22T05:50:42.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f760aec9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76040ae8e0 con 0x7f76040ae500 2024-04-22T05:50:42.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7ffa27fff640 1 -- 172.21.15.40:0/1334608416 shutdown_connections 2024-04-22T05:50:42.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7ffa27fff640 1 --2- 172.21.15.40:0/1334608416 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ffa2002b8c0 0x7ffa200c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7ffa27fff640 1 --2- 172.21.15.40:0/1334608416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa200375e0 0x7ffa200c7df0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7ffa27fff640 1 -- 172.21.15.40:0/1334608416 >> 172.21.15.40:0/1334608416 conn(0x7ffa2003a8a0 msgr2=0x7ffa20036b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f01dda24640 1 -- 172.21.15.40:0/4076492809 >> v2:172.21.15.40:3300/0 conn(0x7f01d80aed40 msgr2=0x7f01d8148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f01dda24640 1 --2- 172.21.15.40:0/4076492809 >> v2:172.21.15.40:3300/0 conn(0x7f01d80aed40 0x7f01d8148730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f01dda24640 1 -- 172.21.15.40:0/4076492809 >> v1:172.21.15.40:6789/0 conn(0x7f01d8073540 legacy=0x7f01d8142380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f01dda24640 1 -- 172.21.15.40:0/4076492809 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d8072290 con 0x7f01d8148c70 2024-04-22T05:50:42.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f760aec9640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7604070940 con 0x7f760406e8e0 2024-04-22T05:50:42.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f7609ec7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76040ae500 0x7f760406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f76096c6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f760406e480 0x7f76040ad210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56026/0 (socket says 172.21.15.40:56026) 2024-04-22T05:50:42.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f76096c6640 1 -- 172.21.15.40:0/2784979554 learned_addr learned my addr 172.21.15.40:0/2784979554 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:50:42.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f01dda24640 1 --2- 172.21.15.40:0/4076492809 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f01d8148c70 0x7f01d814f9c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009870 tx=0x7f01d401ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ab11171d31cbdfc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f7608ec5640 1 -- 172.21.15.40:0/2784979554 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072045236 0 0) 0x7f7604070660 con 0x7f760406e480 2024-04-22T05:50:42.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f7609ec7640 1 -- 172.21.15.40:0/2784979554 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f760406e8e0 msgr2=0x7f76040ad920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:42.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f7609ec7640 1 --2- 172.21.15.40:0/2784979554 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f760406e8e0 0x7f76040ad920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f7609ec7640 1 -- 172.21.15.40:0/2784979554 >> v1:172.21.15.40:6789/0 conn(0x7f760406e480 legacy=0x7f76040ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f7609ec7640 1 -- 172.21.15.40:0/2784979554 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7604070c20 con 0x7f76040ae500 2024-04-22T05:50:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f05ccef8640 1 -- 172.21.15.40:0/3785108589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05c80ad400 msgr2=0x7f05c813a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f05ccef8640 1 --2- 172.21.15.40:0/3785108589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05c80ad400 0x7f05c813a9d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f05c80ada00 tx=0x7f05c001c810 comp rx=0 tx=0).stop 2024-04-22T05:50:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f05ccef8640 1 -- 172.21.15.40:0/3785108589 shutdown_connections 2024-04-22T05:50:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f05ccef8640 1 --2- 172.21.15.40:0/3785108589 >> v2:172.21.15.40:3300/0 conn(0x7f05c81417f0 0x7f05c8148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f05ccef8640 1 --2- 172.21.15.40:0/3785108589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05c80ad400 0x7f05c813a9d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f05ccef8640 1 -- 172.21.15.40:0/3785108589 >> 172.21.15.40:0/3785108589 conn(0x7f05c80b27a0 msgr2=0x7f05c80b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7fe2ceffd640 1 --2- 172.21.15.40:0/1611800090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2d0070dc0 0x7fe2d01512d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40097c0 tx=0x7fe2c401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a837bce325e3b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7fe2ce7fc640 1 -- 172.21.15.40:0/1611800090 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40a6040 con 0x7fe2d0070dc0 2024-04-22T05:50:42.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7fe2ce7fc640 1 -- 172.21.15.40:0/1611800090 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c40aa600 con 0x7fe2d0070dc0 2024-04-22T05:50:42.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7fe2ce7fc640 1 -- 172.21.15.40:0/1611800090 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40aa990 con 0x7fe2d0070dc0 2024-04-22T05:50:42.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f9bea974640 1 Processor -- start 2024-04-22T05:50:42.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7ffa27fff640 1 -- 172.21.15.40:0/1334608416 shutdown_connections 2024-04-22T05:50:42.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f05ccef8640 1 -- 172.21.15.40:0/3785108589 shutdown_connections 2024-04-22T05:50:42.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a1609b640 1 -- 172.21.15.40:0/3227459380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29fc0047d0 msgr2=0x7f29fc004c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:42.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a1609b640 1 --2- 172.21.15.40:0/3227459380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29fc0047d0 0x7f29fc004c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a1609b640 1 -- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29fc003980 msgr2=0x7f29fc003d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:42.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a1609b640 1 --2- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29fc003980 0x7f29fc003d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a1609b640 1 -- 172.21.15.40:0/3227459380 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc003530 con 0x7f29fc0052b0 2024-04-22T05:50:42.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f2a1609b640 1 -- 172.21.15.40:0/3227459380 reap_dead start 2024-04-22T05:50:42.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f9bea974640 1 -- start start 2024-04-22T05:50:42.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa27fff640 1 -- 172.21.15.40:0/1334608416 wait complete. 2024-04-22T05:50:42.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f05ccef8640 1 -- 172.21.15.40:0/3785108589 wait complete. 2024-04-22T05:50:42.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f01cffff640 1 -- 172.21.15.40:0/4076492809 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d40a8a30 con 0x7f01d8148c70 2024-04-22T05:50:42.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f01cffff640 1 -- 172.21.15.40:0/4076492809 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d40a8b90 con 0x7f01d8148c70 2024-04-22T05:50:42.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f01cffff640 1 -- 172.21.15.40:0/4076492809 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d40b0440 con 0x7f01d8148c70 2024-04-22T05:50:42.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f2a1609b640 1 --2- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f29fc0052b0 0x7f29fc00b990 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2a1004efc0 tx=0x7f2a10084a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d995258b3756ac80 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f2a14898640 1 -- 172.21.15.40:0/3227459380 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1010ce80 con 0x7f29fc0052b0 2024-04-22T05:50:42.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f2a14898640 1 -- 172.21.15.40:0/3227459380 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a10114440 con 0x7f29fc0052b0 2024-04-22T05:50:42.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7fe2d4d80640 1 -- 172.21.15.40:0/1611800090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2d0070dc0 msgr2=0x7fe2d01512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7fe2d4d80640 1 --2- 172.21.15.40:0/1611800090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2d0070dc0 0x7fe2d01512d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40097c0 tx=0x7fe2c401c4a0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7fe2d4d80640 1 -- 172.21.15.40:0/1611800090 shutdown_connections 2024-04-22T05:50:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7fe2d4d80640 1 --2- 172.21.15.40:0/1611800090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2d0070dc0 0x7fe2d01512d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7fe2d4d80640 1 --2- 172.21.15.40:0/1611800090 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2d00709e0 0x7fe2d00ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7fe2d4d80640 1 -- 172.21.15.40:0/1611800090 >> 172.21.15.40:0/1611800090 conn(0x7fe2d0073af0 msgr2=0x7fe2d00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7fe2d4d80640 1 -- 172.21.15.40:0/1611800090 shutdown_connections 2024-04-22T05:50:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7fe2d4d80640 1 -- 172.21.15.40:0/1611800090 wait complete. 2024-04-22T05:50:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05ccef8640 1 Processor -- start 2024-04-22T05:50:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa27fff640 1 Processor -- start 2024-04-22T05:50:42.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f01de225640 1 -- 172.21.15.40:0/4076492809 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f01d8148c70 msgr2=0x7f01d814f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f01de225640 1 --2- 172.21.15.40:0/4076492809 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f01d8148c70 0x7f01d814f9c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009870 tx=0x7f01d401ed30 comp rx=0 tx=0).stop 2024-04-22T05:50:42.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f2a1689c640 1 -- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f29fc0052b0 msgr2=0x7f29fc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f2a1689c640 1 --2- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f29fc0052b0 0x7f29fc00b990 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2a1004efc0 tx=0x7f2a10084a60 comp rx=0 tx=0).stop 2024-04-22T05:50:42.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f9bea974640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9bdc031ba0 0x7f9bdc033c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f9bea974640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bdc032ee0 0x7f9bdc0341d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05ccef8640 1 -- start start 2024-04-22T05:50:42.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f9bea974640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bdc0073d0 con 0x7f9bdc006ff0 2024-04-22T05:50:42.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f9bea974640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bdc033890 con 0x7f9bdc031ba0 2024-04-22T05:50:42.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f9bea974640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc035130 con 0x7f9bdc032ee0 2024-04-22T05:50:42.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05ccef8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05c80ad400 0x7f05c81e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05ccef8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c813af10 0x7f05c80ab720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05ccef8640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f05c81417f0 0x7f05c80abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05ccef8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c80a9fc0 con 0x7f05c813af10 2024-04-22T05:50:42.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05ccef8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81e0200 con 0x7f05c80ad400 2024-04-22T05:50:42.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05ccef8640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c80aab20 con 0x7f05c81417f0 2024-04-22T05:50:42.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f7609ec7640 1 --2- 172.21.15.40:0/2784979554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76040ae500 0x7f760406f280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0097c0 tx=0x7f75fc01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8477789c6d430483 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f01de225640 1 -- 172.21.15.40:0/4076492809 shutdown_connections 2024-04-22T05:50:42.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f01de225640 1 --2- 172.21.15.40:0/4076492809 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f01d8148c70 0x7f01d814f9c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f01de225640 1 --2- 172.21.15.40:0/4076492809 >> v2:172.21.15.40:3300/0 conn(0x7f01d80aed40 0x7f01d8148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f01de225640 1 -- 172.21.15.40:0/4076492809 >> 172.21.15.40:0/4076492809 conn(0x7f01d80af950 msgr2=0x7f01d80afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f01de225640 1 -- 172.21.15.40:0/4076492809 shutdown_connections 2024-04-22T05:50:42.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f01de225640 1 -- 172.21.15.40:0/4076492809 wait complete. 2024-04-22T05:50:42.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f2a1689c640 1 -- 172.21.15.40:0/3227459380 shutdown_connections 2024-04-22T05:50:42.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f2a1689c640 1 --2- 172.21.15.40:0/3227459380 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f29fc0052b0 0x7f29fc00b990 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f2a1689c640 1 -- 172.21.15.40:0/3227459380 >> 172.21.15.40:0/3227459380 conn(0x7f2a08005ec0 msgr2=0x7f2a080062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05c7fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f05c81417f0 0x7f05c80abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05c7fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f05c81417f0 0x7f05c80abc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:34998/0 (socket says 172.21.15.40:34998) 2024-04-22T05:50:42.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05c7fff640 1 -- 172.21.15.40:0/3436946160 learned_addr learned my addr 172.21.15.40:0/3436946160 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f9be9972640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9bdc006ff0 0x7f9bdc031490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:58356/0 (socket says 172.21.15.40:58356) 2024-04-22T05:50:42.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f9be9972640 1 -- 172.21.15.40:0/1824625893 learned_addr learned my addr 172.21.15.40:0/1824625893 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:50:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f412359e640 1 Processor -- start 2024-04-22T05:50:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f412359e640 1 -- start start 2024-04-22T05:50:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f412359e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f412406f3b0 0x7f412406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f412359e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4124071f70 0x7f4124137550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f412359e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41240ae1b0 con 0x7f4124137a90 2024-04-22T05:50:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f412359e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f412413d960 con 0x7f4124071f70 2024-04-22T05:50:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.274+0000 7f412359e640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4124071870 con 0x7f412406f3b0 2024-04-22T05:50:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f4121d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4124071f70 0x7f4124137550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f4121d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4124071f70 0x7f4124137550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59480/0 (socket says 172.21.15.40:59480) 2024-04-22T05:50:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f4121d9b640 1 -- 172.21.15.40:0/2711741130 learned_addr learned my addr 172.21.15.40:0/2711741130 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f412259c640 1 --2- 172.21.15.40:0/2711741130 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f412406f3b0 0x7f412406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f4121d9b640 1 -- 172.21.15.40:0/2711741130 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f412406f3b0 msgr2=0x7f412406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa27fff640 1 -- start start 2024-04-22T05:50:42.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f4121d9b640 1 --2- 172.21.15.40:0/2711741130 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f412406f3b0 0x7f412406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f4121d9b640 1 -- 172.21.15.40:0/2711741130 >> v1:172.21.15.40:6789/0 conn(0x7f4124137a90 legacy=0x7f412413ddb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:42.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f4121d9b640 1 -- 172.21.15.40:0/2711741130 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4124071b50 con 0x7f4124071f70 2024-04-22T05:50:42.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa27fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa2002b8c0 0x7ffa200c2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa27fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa2002c5b0 0x7ffa200be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa27fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ffa200375e0 0x7ffa201671b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa27fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa200c4120 con 0x7ffa2002c5b0 2024-04-22T05:50:42.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa27fff640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa200c4390 con 0x7ffa2002b8c0 2024-04-22T05:50:42.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05c6ffd640 1 --2- 172.21.15.40:0/3436946160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c813af10 0x7f05c80ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa27fff640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa20035c20 con 0x7ffa200375e0 2024-04-22T05:50:42.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa26ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa2002b8c0 0x7ffa200c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa26ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa2002b8c0 0x7ffa200c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:58746/0 (socket says 172.21.15.40:58746) 2024-04-22T05:50:42.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa26ffd640 1 -- 172.21.15.40:0/3001873620 learned_addr learned my addr 172.21.15.40:0/3001873620 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f7608ec5640 1 -- 172.21.15.40:0/2784979554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc0ab400 con 0x7f76040ae500 2024-04-22T05:50:42.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f7608ec5640 1 -- 172.21.15.40:0/2784979554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75fc0a7050 con 0x7f76040ae500 2024-04-22T05:50:42.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f7608ec5640 1 -- 172.21.15.40:0/2784979554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc0ab780 con 0x7f76040ae500 2024-04-22T05:50:42.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7fe2d4d80640 1 Processor -- start 2024-04-22T05:50:42.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f9be9171640 1 --2- 172.21.15.40:0/1824625893 >> v2:172.21.15.40:3300/0 conn(0x7f9bdc031ba0 0x7f9bdc033c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f9be8970640 1 -- 172.21.15.40:0/1824625893 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700710065 0 0) 0x7f9bdc0073d0 con 0x7f9bdc006ff0 2024-04-22T05:50:42.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05c7fff640 1 -- 172.21.15.40:0/3436946160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05c80ad400 msgr2=0x7f05c81e06e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T05:50:42.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05c7fff640 1 --2- 172.21.15.40:0/3436946160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05c80ad400 0x7f05c81e06e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05c7fff640 1 -- 172.21.15.40:0/3436946160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c813af10 msgr2=0x7f05c80ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05c7fff640 1 --2- 172.21.15.40:0/3436946160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c813af10 0x7f05c80ab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa267fc640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa2002c5b0 0x7ffa200be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7ffa277fe640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ffa200375e0 0x7ffa201671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f9be8970640 1 -- 172.21.15.40:0/1824625893 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bd0003530 con 0x7f9bdc006ff0 2024-04-22T05:50:42.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05c7fff640 1 -- 172.21.15.40:0/3436946160 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c0009a70 con 0x7f05c81417f0 2024-04-22T05:50:42.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f01de225640 1 Processor -- start 2024-04-22T05:50:42.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f01de225640 1 -- start start 2024-04-22T05:50:42.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7f05c7fff640 1 --2- 172.21.15.40:0/3436946160 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f05c81417f0 0x7f05c80abc60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f05b800e7f0 tx=0x7f05b800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7ffa277fe640 1 -- 172.21.15.40:0/3001873620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa2002b8c0 msgr2=0x7ffa200c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7ffa277fe640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa2002b8c0 0x7ffa200c2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7ffa277fe640 1 -- 172.21.15.40:0/3001873620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa2002c5b0 msgr2=0x7ffa200be8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7ffa277fe640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa2002c5b0 0x7ffa200be8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7ffa277fe640 1 -- 172.21.15.40:0/3001873620 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa180094d0 con 0x7ffa200375e0 2024-04-22T05:50:42.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f760aec9640 1 -- 172.21.15.40:0/2784979554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76040ae500 msgr2=0x7f760406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f760aec9640 1 --2- 172.21.15.40:0/2784979554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76040ae500 0x7f760406f280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0097c0 tx=0x7f75fc01c380 comp rx=0 tx=0).stop 2024-04-22T05:50:42.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.278+0000 7fe2d4d80640 1 -- start start 2024-04-22T05:50:42.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2d4d80640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2d00709e0 0x7fe2d00aa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2d4d80640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2d0070dc0 0x7fe2d00aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2d4d80640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2d0071200 0x7fe2d0088430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2d4d80640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00ab360 con 0x7fe2d0071200 2024-04-22T05:50:42.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2d4d80640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00a7d70 con 0x7fe2d00709e0 2024-04-22T05:50:42.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2d4d80640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d00a96d0 con 0x7fe2d0070dc0 2024-04-22T05:50:42.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f760aec9640 1 -- 172.21.15.40:0/2784979554 shutdown_connections 2024-04-22T05:50:42.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f760aec9640 1 --2- 172.21.15.40:0/2784979554 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f760406e8e0 0x7f76040ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f760aec9640 1 --2- 172.21.15.40:0/2784979554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76040ae500 0x7f760406f280 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0097c0 tx=0x7f75fc01c380 comp rx=0 tx=0).stop 2024-04-22T05:50:42.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f760aec9640 1 -- 172.21.15.40:0/2784979554 >> 172.21.15.40:0/2784979554 conn(0x7f76040b29e0 msgr2=0x7f76040b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7ffa277fe640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ffa200375e0 0x7ffa201671b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffa28063b90 tx=0x7ffa28067b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2cf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2d00709e0 0x7fe2d00aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2cf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2d00709e0 0x7fe2d00aa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:58748/0 (socket says 172.21.15.40:58748) 2024-04-22T05:50:42.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2cf7fe640 1 -- 172.21.15.40:0/2210182230 learned_addr learned my addr 172.21.15.40:0/2210182230 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2ceffd640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2d0070dc0 0x7fe2d00aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2cffff640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2d0071200 0x7fe2d0088430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01de225640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d8073540 0x7f01d8145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01de225640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f01d80aed40 0x7f01d80a7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01de225640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01d8148c70 0x7f01d813c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01de225640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8145070 con 0x7f01d8073540 2024-04-22T05:50:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01de225640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80b2b30 con 0x7f01d8148c70 2024-04-22T05:50:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01de225640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d81452c0 con 0x7f01d80aed40 2024-04-22T05:50:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01dd223640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d8073540 0x7f01d8145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01dd223640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d8073540 0x7f01d8145b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59532/0 (socket says 172.21.15.40:59532) 2024-04-22T05:50:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01dd223640 1 -- 172.21.15.40:0/2846142691 learned_addr learned my addr 172.21.15.40:0/2846142691 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01dda24640 1 --2- 172.21.15.40:0/2846142691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01d8148c70 0x7f01d813c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2cf7fe640 1 -- 172.21.15.40:0/2210182230 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2d0070dc0 msgr2=0x7fe2d00aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2cf7fe640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe2d0070dc0 0x7fe2d00aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2cf7fe640 1 -- 172.21.15.40:0/2210182230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2d0071200 msgr2=0x7fe2d0088430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2cf7fe640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2d0071200 0x7fe2d0088430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7fe2cf7fe640 1 -- 172.21.15.40:0/2210182230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c4009420 con 0x7fe2d00709e0 2024-04-22T05:50:42.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01dd223640 1 -- 172.21.15.40:0/2846142691 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f01d80aed40 msgr2=0x7f01d80a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f760aec9640 1 -- 172.21.15.40:0/2784979554 shutdown_connections 2024-04-22T05:50:42.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01dd223640 1 --2- 172.21.15.40:0/2846142691 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f01d80aed40 0x7f01d80a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01dd223640 1 -- 172.21.15.40:0/2846142691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01d8148c70 msgr2=0x7f01d813c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01dd223640 1 --2- 172.21.15.40:0/2846142691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01d8148c70 0x7f01d813c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01dd223640 1 -- 172.21.15.40:0/2846142691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d40094d0 con 0x7f01d8073540 2024-04-22T05:50:42.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f760aec9640 1 -- 172.21.15.40:0/2784979554 wait complete. 2024-04-22T05:50:42.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f2a1689c640 1 -- 172.21.15.40:0/3227459380 shutdown_connections 2024-04-22T05:50:42.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01dd223640 1 --2- 172.21.15.40:0/2846142691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d8073540 0x7f01d8145b30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c7f0 tx=0x7f01d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7ffa0f7fe640 1 -- 172.21.15.40:0/3001873620 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2806f870 con 0x7ffa200375e0 2024-04-22T05:50:42.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7ffa0f7fe640 1 -- 172.21.15.40:0/3001873620 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2806f9d0 con 0x7ffa200375e0 2024-04-22T05:50:42.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f9be9171640 1 -- 172.21.15.40:0/1824625893 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bdc032ee0 msgr2=0x7f9bdc0341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f9be9171640 1 --2- 172.21.15.40:0/1824625893 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bdc032ee0 0x7f9bdc0341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f9be9171640 1 -- 172.21.15.40:0/1824625893 >> v1:172.21.15.62:6789/0 conn(0x7f9bdc006ff0 legacy=0x7f9bdc031490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f9be9171640 1 -- 172.21.15.40:0/1824625893 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc0353a0 con 0x7f9bdc031ba0 2024-04-22T05:50:42.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f05affff640 1 -- 172.21.15.40:0/3436946160 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b800eea0 con 0x7f05c81417f0 2024-04-22T05:50:42.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f05affff640 1 -- 172.21.15.40:0/3436946160 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b800c940 con 0x7f05c81417f0 2024-04-22T05:50:42.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f05affff640 1 -- 172.21.15.40:0/3436946160 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b800ccd0 con 0x7f05c81417f0 2024-04-22T05:50:42.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f05ccef8640 1 -- 172.21.15.40:0/3436946160 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c80aacf0 con 0x7f05c81417f0 2024-04-22T05:50:42.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f05ccef8640 1 -- 172.21.15.40:0/3436946160 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c80a8430 con 0x7f05c81417f0 2024-04-22T05:50:42.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f05ccef8640 1 -- 172.21.15.40:0/3436946160 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f05c8071810 con 0x7f05c81417f0 2024-04-22T05:50:42.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f4121d9b640 1 --2- 172.21.15.40:0/2711741130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4124071f70 0x7f4124137550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f411c009e50 tx=0x7f411c01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d12d39429b79ce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01cdffb640 1 -- 172.21.15.40:0/2846142691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0003b80 con 0x7f01d8073540 2024-04-22T05:50:42.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01de225640 1 -- 172.21.15.40:0/2846142691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d813c560 con 0x7f01d8073540 2024-04-22T05:50:42.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01de225640 1 -- 172.21.15.40:0/2846142691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d8143080 con 0x7f01d8073540 2024-04-22T05:50:42.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7ffa0f7fe640 1 -- 172.21.15.40:0/3001873620 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2806fd80 con 0x7ffa200375e0 2024-04-22T05:50:42.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f9be9171640 1 --2- 172.21.15.40:0/1824625893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bdc031ba0 0x7f9bdc033c90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40097c0 tx=0x7f9bd401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7310f522f0598364 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01cdffb640 1 -- 172.21.15.40:0/2846142691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d0003ce0 con 0x7f01d8073540 2024-04-22T05:50:42.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01cdffb640 1 -- 172.21.15.40:0/2846142691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d000aa90 con 0x7f01d8073540 2024-04-22T05:50:42.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f760aec9640 1 Processor -- start 2024-04-22T05:50:42.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f2a1689c640 1 -- 172.21.15.40:0/3227459380 wait complete. 2024-04-22T05:50:42.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f2a1689c640 1 Processor -- start 2024-04-22T05:50:42.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f01de225640 1 -- 172.21.15.40:0/2846142691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f01b8000f20 con 0x7f01d8073540 2024-04-22T05:50:42.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f412159a640 1 -- 172.21.15.40:0/2711741130 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c0a5640 con 0x7f4124071f70 2024-04-22T05:50:42.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f412159a640 1 -- 172.21.15.40:0/2711741130 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c0a57a0 con 0x7f4124071f70 2024-04-22T05:50:42.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f412159a640 1 -- 172.21.15.40:0/2711741130 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c0a5b30 con 0x7f4124071f70 2024-04-22T05:50:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7ffa27fff640 1 -- 172.21.15.40:0/3001873620 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa20036890 con 0x7ffa200375e0 2024-04-22T05:50:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7ffa27fff640 1 -- 172.21.15.40:0/3001873620 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa201678c0 con 0x7ffa200375e0 2024-04-22T05:50:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f760aec9640 1 -- start start 2024-04-22T05:50:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7ffa27fff640 1 -- 172.21.15.40:0/3001873620 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa200c4500 con 0x7ffa200375e0 2024-04-22T05:50:42.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f760aec9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760406e480 0x7f760415e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f760aec9640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f760406e8e0 0x7f7604156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f760aec9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f760415bc60 0x7f760415c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f760aec9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7604157f40 con 0x7f760406e480 2024-04-22T05:50:42.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f760aec9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76041581b0 con 0x7f760415bc60 2024-04-22T05:50:42.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f760aec9640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76041573d0 con 0x7f760406e8e0 2024-04-22T05:50:42.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.282+0000 7f2a1689c640 1 -- start start 2024-04-22T05:50:42.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a1689c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a080f68b0 0x7f2a080f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a1689c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2a080f5ab0 0x7f2a0818efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a1689c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a0818f4e0 0x7f2a080f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a1689c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a080f71d0 con 0x7f2a0818f4e0 2024-04-22T05:50:42.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a1689c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a080f85a0 con 0x7f2a080f68b0 2024-04-22T05:50:42.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a1689c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a080dfdb0 con 0x7f2a080f5ab0 2024-04-22T05:50:42.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7fe2cf7fe640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2d00709e0 0x7fe2d00aa590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000b340 tx=0x7fe2c0009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f412359e640 1 -- 172.21.15.40:0/2711741130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4124071f70 msgr2=0x7f4124137550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f412359e640 1 --2- 172.21.15.40:0/2711741130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4124071f70 0x7f4124137550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f411c009e50 tx=0x7f411c01c8b0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f7609ec7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760406e480 0x7f760415e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f7609ec7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760406e480 0x7f760415e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59540/0 (socket says 172.21.15.40:59540) 2024-04-22T05:50:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f7609ec7640 1 -- 172.21.15.40:0/2543326422 learned_addr learned my addr 172.21.15.40:0/2543326422 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f05affff640 1 -- 172.21.15.40:0/3436946160 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f05b8013470 con 0x7f05c81417f0 2024-04-22T05:50:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f05affff640 1 --2- 172.21.15.40:0/3436946160 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f05b40666e0 0x7f05b4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f05affff640 1 -- 172.21.15.40:0/3436946160 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f05b8084300 con 0x7f05c81417f0 2024-04-22T05:50:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f76096c6640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f760406e8e0 0x7f7604156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7fe2b7fff640 1 -- 172.21.15.40:0/2210182230 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0009b60 con 0x7fe2d00709e0 2024-04-22T05:50:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7fe2b7fff640 1 -- 172.21.15.40:0/2210182230 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c0009cc0 con 0x7fe2d00709e0 2024-04-22T05:50:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7fe2b7fff640 1 -- 172.21.15.40:0/2210182230 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c00175f0 con 0x7fe2d00709e0 2024-04-22T05:50:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7fe2d4d80640 1 -- 172.21.15.40:0/2210182230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d00a7990 con 0x7fe2d00709e0 2024-04-22T05:50:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7fe2d4d80640 1 -- 172.21.15.40:0/2210182230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d0088b40 con 0x7fe2d00709e0 2024-04-22T05:50:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f05affff640 1 --2- 172.21.15.40:0/3436946160 >> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] conn(0x7f05b406cac0 0x7f05b406eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f05affff640 1 -- 172.21.15.40:0/3436946160 --> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05b406f590 con 0x7f05b406cac0 2024-04-22T05:50:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f05affff640 1 -- 172.21.15.40:0/3436946160 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05b8084670 con 0x7f05c81417f0 2024-04-22T05:50:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f05c6ffd640 1 --2- 172.21.15.40:0/3436946160 >> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] conn(0x7f05b406cac0 0x7f05b406eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f05c77fe640 1 --2- 172.21.15.40:0/3436946160 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f05b40666e0 0x7f05b4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f05c6ffd640 1 --2- 172.21.15.40:0/3436946160 >> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] conn(0x7f05b406cac0 0x7f05b406eee0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f7609ec7640 1 -- 172.21.15.40:0/2543326422 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f760406e8e0 msgr2=0x7f7604156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f7609ec7640 1 --2- 172.21.15.40:0/2543326422 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f760406e8e0 0x7f7604156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f7609ec7640 1 -- 172.21.15.40:0/2543326422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f760415bc60 msgr2=0x7f760415c0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T05:50:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f7609ec7640 1 --2- 172.21.15.40:0/2543326422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f760415bc60 0x7f760415c0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f7609ec7640 1 -- 172.21.15.40:0/2543326422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc009420 con 0x7f760406e480 2024-04-22T05:50:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f7609ec7640 1 --2- 172.21.15.40:0/2543326422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760406e480 0x7f760415e360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75fc019fb0 tx=0x7f75fc019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a1609b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a0818f4e0 0x7f2a080f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a1609b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a0818f4e0 0x7f2a080f8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59542/0 (socket says 172.21.15.40:59542) 2024-04-22T05:50:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a1609b640 1 -- 172.21.15.40:0/2448623042 learned_addr learned my addr 172.21.15.40:0/2448623042 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a1589a640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a080f68b0 0x7f2a080f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a15099640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2a080f5ab0 0x7f2a0818efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a15099640 1 -- 172.21.15.40:0/2448623042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a080f68b0 msgr2=0x7f2a080f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a15099640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a080f68b0 0x7f2a080f6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a15099640 1 -- 172.21.15.40:0/2448623042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a0818f4e0 msgr2=0x7f2a080f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a15099640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a0818f4e0 0x7f2a080f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a15099640 1 -- 172.21.15.40:0/2448623042 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0818f940 con 0x7f2a080f5ab0 2024-04-22T05:50:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a15099640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2a080f5ab0 0x7f2a0818efa0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000b3e0 tx=0x7f2a00009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a067fc640 1 -- 172.21.15.40:0/2448623042 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a00011830 con 0x7f2a080f5ab0 2024-04-22T05:50:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f9be8970640 1 -- 172.21.15.40:0/1824625893 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd40ab600 con 0x7f9bdc031ba0 2024-04-22T05:50:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f9be8970640 1 -- 172.21.15.40:0/1824625893 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd40a7050 con 0x7f9bdc031ba0 2024-04-22T05:50:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f9be8970640 1 -- 172.21.15.40:0/1824625893 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd40ab9c0 con 0x7f9bdc031ba0 2024-04-22T05:50:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f9bea974640 1 -- 172.21.15.40:0/1824625893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bdc031ba0 msgr2=0x7f9bdc033c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f9bea974640 1 --2- 172.21.15.40:0/1824625893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bdc031ba0 0x7f9bdc033c90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40097c0 tx=0x7f9bd401c3a0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7fe2d4d80640 1 -- 172.21.15.40:0/2210182230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe2b0000f20 con 0x7fe2d00709e0 2024-04-22T05:50:42.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f05c77fe640 1 --2- 172.21.15.40:0/3436946160 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f05b40666e0 0x7f05b4068ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f05c00af000 tx=0x7f05c00a7070 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f75fa7fc640 1 -- 172.21.15.40:0/2543326422 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc0b4400 con 0x7f760406e480 2024-04-22T05:50:42.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a067fc640 1 -- 172.21.15.40:0/2448623042 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a00011990 con 0x7f2a080f5ab0 2024-04-22T05:50:42.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a067fc640 1 -- 172.21.15.40:0/2448623042 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a00011d20 con 0x7f2a080f5ab0 2024-04-22T05:50:42.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a1689c640 1 -- 172.21.15.40:0/2448623042 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a080f8fa0 con 0x7f2a080f5ab0 2024-04-22T05:50:42.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a1689c640 1 -- 172.21.15.40:0/2448623042 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a080f9450 con 0x7f2a080f5ab0 2024-04-22T05:50:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f2a1689c640 1 -- 172.21.15.40:0/2448623042 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2a08038cd0 con 0x7f2a080f5ab0 2024-04-22T05:50:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f01cdffb640 1 -- 172.21.15.40:0/2846142691 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f01d000abf0 con 0x7f01d8073540 2024-04-22T05:50:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7f05affff640 1 -- 172.21.15.40:0/3436946160 <== osd.1 v2:172.21.15.40:6800/24984 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f05b406f590 con 0x7f05b406cac0 2024-04-22T05:50:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7ffa0f7fe640 1 -- 172.21.15.40:0/3001873620 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffa2809c3a0 con 0x7ffa200375e0 2024-04-22T05:50:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7ffa0f7fe640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ffa140666e0 0x7ffa14068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7ffa0f7fe640 1 -- 172.21.15.40:0/3001873620 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ffa280fd1c0 con 0x7ffa200375e0 2024-04-22T05:50:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7ffa0f7fe640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] conn(0x7ffa1406c950 0x7ffa1406ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7ffa0f7fe640 1 -- 172.21.15.40:0/3001873620 --> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa1406f420 con 0x7ffa1406c950 2024-04-22T05:50:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.286+0000 7ffa0f7fe640 1 -- 172.21.15.40:0/3001873620 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa280fd530 con 0x7ffa200375e0 2024-04-22T05:50:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f75fa7fc640 1 -- 172.21.15.40:0/2543326422 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75fc0b4560 con 0x7f760406e480 2024-04-22T05:50:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f760aec9640 1 -- 172.21.15.40:0/2543326422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76041570a0 con 0x7f760406e480 2024-04-22T05:50:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f75fa7fc640 1 -- 172.21.15.40:0/2543326422 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc0b48d0 con 0x7f760406e480 2024-04-22T05:50:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f760aec9640 1 -- 172.21.15.40:0/2543326422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f760415c7d0 con 0x7f760406e480 2024-04-22T05:50:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7fe2b7fff640 1 -- 172.21.15.40:0/2210182230 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe2c00059a0 con 0x7fe2d00709e0 2024-04-22T05:50:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7fe2b7fff640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe2bc0666c0 0x7fe2bc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7fe2b7fff640 1 -- 172.21.15.40:0/2210182230 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe2c0084ab0 con 0x7fe2d00709e0 2024-04-22T05:50:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7fe2b7fff640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] conn(0x7fe2bc06caa0 0x7fe2bc06eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7fe2b7fff640 1 -- 172.21.15.40:0/2210182230 --> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2bc06f570 con 0x7fe2bc06caa0 2024-04-22T05:50:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7fe2b7fff640 1 -- 172.21.15.40:0/2210182230 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2c0084e60 con 0x7fe2d00709e0 2024-04-22T05:50:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7ffa267fc640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] conn(0x7ffa1406c950 0x7ffa1406ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7ffa26ffd640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ffa140666e0 0x7ffa14068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7ffa267fc640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] conn(0x7ffa1406c950 0x7ffa1406ed70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f9bea974640 1 -- 172.21.15.40:0/1824625893 shutdown_connections 2024-04-22T05:50:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f9bea974640 1 --2- 172.21.15.40:0/1824625893 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bdc032ee0 0x7f9bdc0341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f9bea974640 1 --2- 172.21.15.40:0/1824625893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bdc031ba0 0x7f9bdc033c90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f9bea974640 1 -- 172.21.15.40:0/1824625893 >> 172.21.15.40:0/1824625893 conn(0x7f9bdc006270 msgr2=0x7f9bdc006680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f9bea974640 1 -- 172.21.15.40:0/1824625893 shutdown_connections 2024-04-22T05:50:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f412359e640 1 -- 172.21.15.40:0/2711741130 shutdown_connections 2024-04-22T05:50:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f412359e640 1 --2- 172.21.15.40:0/2711741130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4124071f70 0x7f4124137550 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f412359e640 1 --2- 172.21.15.40:0/2711741130 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f412406f3b0 0x7f412406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f412359e640 1 -- 172.21.15.40:0/2711741130 >> 172.21.15.40:0/2711741130 conn(0x7f41240b3980 msgr2=0x7f41240b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f01cdffb640 1 --2- 172.21.15.40:0/2846142691 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f01bc066690 0x7f01bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f01cdffb640 1 -- 172.21.15.40:0/2846142691 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f01d0084830 con 0x7f01d8073540 2024-04-22T05:50:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f01cdffb640 1 --2- 172.21.15.40:0/2846142691 >> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] conn(0x7f01bc06ca70 0x7f01bc06ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f01cdffb640 1 -- 172.21.15.40:0/2846142691 --> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f01bc06f540 con 0x7f01bc06ca70 2024-04-22T05:50:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7fe2cffff640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] conn(0x7fe2bc06caa0 0x7fe2bc06eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f01dca22640 1 --2- 172.21.15.40:0/2846142691 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f01bc066690 0x7f01bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f01dda24640 1 --2- 172.21.15.40:0/2846142691 >> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] conn(0x7f01bc06ca70 0x7f01bc06ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7fe2ceffd640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe2bc0666c0 0x7fe2bc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7fe2cffff640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] conn(0x7fe2bc06caa0 0x7fe2bc06eec0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f01cdffb640 1 -- 172.21.15.40:0/2846142691 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01d000aea0 con 0x7f01d8073540 2024-04-22T05:50:42.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f01dda24640 1 --2- 172.21.15.40:0/2846142691 >> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] conn(0x7f01bc06ca70 0x7f01bc06ee90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f75fa7fc640 1 -- 172.21.15.40:0/2543326422 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f75fc0b2040 con 0x7f760406e480 2024-04-22T05:50:42.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7fe2ceffd640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe2bc0666c0 0x7fe2bc068b80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40ae000 tx=0x7fe2c40b5040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7ffa0f7fe640 1 -- 172.21.15.40:0/3001873620 <== osd.2 v2:172.21.15.40:6808/24986 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ffa1406f420 con 0x7ffa1406c950 2024-04-22T05:50:42.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f01dca22640 1 --2- 172.21.15.40:0/2846142691 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f01bc066690 0x7f01bc068b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f01d80a89e0 tx=0x7f01c8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7fe2b7fff640 1 -- 172.21.15.40:0/2210182230 <== osd.0 v2:172.21.15.40:6816/24983 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fe2bc06f570 con 0x7fe2bc06caa0 2024-04-22T05:50:42.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7ffa26ffd640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ffa140666e0 0x7ffa14068ba0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffa1801aaa0 tx=0x7ffa180a6000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f9bea974640 1 -- 172.21.15.40:0/1824625893 wait complete. 2024-04-22T05:50:42.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f9bea974640 1 Processor -- start 2024-04-22T05:50:42.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f75fa7fc640 1 --2- 172.21.15.40:0/2543326422 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f75e8066690 0x7f75e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f75fa7fc640 1 -- 172.21.15.40:0/2543326422 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f75fc11e0f0 con 0x7f760406e480 2024-04-22T05:50:42.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f412359e640 1 -- 172.21.15.40:0/2711741130 shutdown_connections 2024-04-22T05:50:42.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f412359e640 1 -- 172.21.15.40:0/2711741130 wait complete. 2024-04-22T05:50:42.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f76096c6640 1 --2- 172.21.15.40:0/2543326422 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f75e8066690 0x7f75e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.290+0000 7f2a067fc640 1 -- 172.21.15.40:0/2448623042 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2a0001a400 con 0x7f2a080f5ab0 2024-04-22T05:50:42.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f760aec9640 1 --2- 172.21.15.40:0/2543326422 >> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] conn(0x7f76041f9950 0x7f76041f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f2a067fc640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f29f4066790 0x7f29f4068c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f2a067fc640 1 -- 172.21.15.40:0/2448623042 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2a00084520 con 0x7f2a080f5ab0 2024-04-22T05:50:42.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f01cdffb640 1 -- 172.21.15.40:0/2846142691 <== osd.7 v2:172.21.15.62:6808/25138 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f01bc06f540 con 0x7f01bc06ca70 2024-04-22T05:50:42.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f760a6c8640 1 --2- 172.21.15.40:0/2543326422 >> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] conn(0x7f76041f9950 0x7f76041f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f2a067fc640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] conn(0x7f29f406cb70 0x7f29f406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f2a067fc640 1 -- 172.21.15.40:0/2448623042 --> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29f406f640 con 0x7f29f406cb70 2024-04-22T05:50:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f2a067fc640 1 -- 172.21.15.40:0/2448623042 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a000848d0 con 0x7f2a080f5ab0 2024-04-22T05:50:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f2a1609b640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] conn(0x7f29f406cb70 0x7f29f406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412359e640 1 Processor -- start 2024-04-22T05:50:42.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9bea974640 1 -- start start 2024-04-22T05:50:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f2a1589a640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f29f4066790 0x7f29f4068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f760a6c8640 1 --2- 172.21.15.40:0/2543326422 >> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] conn(0x7f76041f9950 0x7f76041f9d30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412359e640 1 -- start start 2024-04-22T05:50:42.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f760aec9640 1 -- 172.21.15.40:0/2543326422 --> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f760415b980 con 0x7f76041f9950 2024-04-22T05:50:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412359e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f412406f3b0 0x7f41240a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9bea974640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bdc006ff0 0x7f9bdc02b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9bea974640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bdc031ba0 0x7f9bdc0c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9bea974640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bdc032ee0 0x7f9bdc008d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9bea974640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc0c4020 con 0x7f9bdc031ba0 2024-04-22T05:50:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9bea974640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc0099d0 con 0x7f9bdc032ee0 2024-04-22T05:50:42.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9bea974640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc0c3e80 con 0x7f9bdc006ff0 2024-04-22T05:50:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f76096c6640 1 --2- 172.21.15.40:0/2543326422 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f75e8066690 0x7f75e8068b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f760415e8a0 tx=0x7f75f4007b60 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f2a1609b640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] conn(0x7f29f406cb70 0x7f29f406ef90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f2a1589a640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f29f4066790 0x7f29f4068c50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c000fe0 tx=0x7f2a0c0015a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9be9972640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bdc006ff0 0x7f9bdc02b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9be9972640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bdc006ff0 0x7f9bdc02b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35062/0 (socket says 172.21.15.40:35062) 2024-04-22T05:50:42.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9be9972640 1 -- 172.21.15.40:0/2693920703 learned_addr learned my addr 172.21.15.40:0/2693920703 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9bea173640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bdc032ee0 0x7f9bdc008d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4124071f70 0x7f4124135430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412359e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4124137a90 0x7f4124135970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412413b270 con 0x7f4124071f70 2024-04-22T05:50:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412359e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412413b690 con 0x7f412406f3b0 2024-04-22T05:50:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412359e640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41240a8c00 con 0x7f4124137a90 2024-04-22T05:50:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f412406f3b0 0x7f41240a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f412406f3b0 0x7f41240a77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:58786/0 (socket says 172.21.15.40:58786) 2024-04-22T05:50:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412259c640 1 -- 172.21.15.40:0/4099747399 learned_addr learned my addr 172.21.15.40:0/4099747399 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f4121d9b640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4124071f70 0x7f4124135430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f75fa7fc640 1 -- 172.21.15.40:0/2543326422 <== osd.3 v2:172.21.15.40:6817/24985 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f760415b980 con 0x7f76041f9950 2024-04-22T05:50:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f4122d9d640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4124137a90 0x7f4124135970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9bea173640 1 -- 172.21.15.40:0/2693920703 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bdc006ff0 msgr2=0x7f9bdc02b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9bea173640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9bdc006ff0 0x7f9bdc02b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9bea173640 1 -- 172.21.15.40:0/2693920703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bdc031ba0 msgr2=0x7f9bdc0c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f9be9171640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bdc031ba0 0x7f9bdc0c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412259c640 1 -- 172.21.15.40:0/4099747399 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4124137a90 msgr2=0x7f4124135970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412259c640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4124137a90 0x7f4124135970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412259c640 1 -- 172.21.15.40:0/4099747399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4124071f70 msgr2=0x7f4124135430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412259c640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4124071f70 0x7f4124135430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.294+0000 7f412259c640 1 -- 172.21.15.40:0/4099747399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c009b30 con 0x7f412406f3b0 2024-04-22T05:50:42.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f9bea173640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bdc031ba0 0x7f9bdc0c3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f9bea173640 1 -- 172.21.15.40:0/2693920703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd4009420 con 0x7f9bdc032ee0 2024-04-22T05:50:42.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f9be9171640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bdc031ba0 0x7f9bdc0c3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:50:42.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f2a067fc640 1 -- 172.21.15.40:0/2448623042 <== osd.4 v2:172.21.15.62:6816/25135 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f29f406f640 con 0x7f29f406cb70 2024-04-22T05:50:42.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f412259c640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f412406f3b0 0x7f41240a77d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f411400e870 tx=0x7f411400ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f9bea173640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bdc032ee0 0x7f9bdc008d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9be4063b90 tx=0x7f9be4067200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f4112ffd640 1 -- 172.21.15.40:0/4099747399 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411400eec0 con 0x7f412406f3b0 2024-04-22T05:50:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f4112ffd640 1 -- 172.21.15.40:0/4099747399 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411400c940 con 0x7f412406f3b0 2024-04-22T05:50:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f4112ffd640 1 -- 172.21.15.40:0/4099747399 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411400ccb0 con 0x7f412406f3b0 2024-04-22T05:50:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f412359e640 1 -- 172.21.15.40:0/4099747399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41240a8e70 con 0x7f412406f3b0 2024-04-22T05:50:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f412359e640 1 -- 172.21.15.40:0/4099747399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412413ad90 con 0x7f412406f3b0 2024-04-22T05:50:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f412359e640 1 -- 172.21.15.40:0/4099747399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f412404e170 con 0x7f412406f3b0 2024-04-22T05:50:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f9bda7fc640 1 -- 172.21.15.40:0/2693920703 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be406f890 con 0x7f9bdc032ee0 2024-04-22T05:50:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f9bda7fc640 1 -- 172.21.15.40:0/2693920703 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be406f9f0 con 0x7f9bdc032ee0 2024-04-22T05:50:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f9bda7fc640 1 -- 172.21.15.40:0/2693920703 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be406fe20 con 0x7f9bdc032ee0 2024-04-22T05:50:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f9bea974640 1 -- 172.21.15.40:0/2693920703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bdc0c57f0 con 0x7f9bdc032ee0 2024-04-22T05:50:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f9bea974640 1 -- 172.21.15.40:0/2693920703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bdc0c5d20 con 0x7f9bdc032ee0 2024-04-22T05:50:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.298+0000 7f9bea974640 1 -- 172.21.15.40:0/2693920703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9bdc0c2220 con 0x7f9bdc032ee0 2024-04-22T05:50:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f4112ffd640 1 -- 172.21.15.40:0/4099747399 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4114013470 con 0x7f412406f3b0 2024-04-22T05:50:42.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f4112ffd640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f41040666c0 0x7f4104068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f4112ffd640 1 -- 172.21.15.40:0/4099747399 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f41140843a0 con 0x7f412406f3b0 2024-04-22T05:50:42.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f4112ffd640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] conn(0x7f410406caa0 0x7f410406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f4112ffd640 1 -- 172.21.15.40:0/4099747399 --> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f410406f570 con 0x7f410406caa0 2024-04-22T05:50:42.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f4112ffd640 1 -- 172.21.15.40:0/4099747399 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4114084750 con 0x7f412406f3b0 2024-04-22T05:50:42.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f9bda7fc640 1 -- 172.21.15.40:0/2693920703 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9be409c3a0 con 0x7f9bdc032ee0 2024-04-22T05:50:42.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f9bda7fc640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9bc8066760 0x7f9bc8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f9bda7fc640 1 -- 172.21.15.40:0/2693920703 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9be40fd1b0 con 0x7f9bdc032ee0 2024-04-22T05:50:42.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f9bda7fc640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] conn(0x7f9bc806cb40 0x7f9bc806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f9bda7fc640 1 -- 172.21.15.40:0/2693920703 --> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9bc806f610 con 0x7f9bc806cb40 2024-04-22T05:50:42.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f9bda7fc640 1 -- 172.21.15.40:0/2693920703 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9be40fd560 con 0x7f9bdc032ee0 2024-04-22T05:50:42.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f9be9972640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9bc8066760 0x7f9bc8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f4121d9b640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f41040666c0 0x7f4104068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f4122d9d640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] conn(0x7f410406caa0 0x7f410406eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.302+0000 7f9be9171640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] conn(0x7f9bc806cb40 0x7f9bc806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.306+0000 7f9be9972640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9bc8066760 0x7f9bc8068c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9be00011f0 tx=0x7f9be0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.306+0000 7f4121d9b640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f41040666c0 0x7f4104068b80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f411c0ae000 tx=0x7f411c0a7070 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.306+0000 7f4122d9d640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] conn(0x7f410406caa0 0x7f410406eec0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.306+0000 7f9be9171640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] conn(0x7f9bc806cb40 0x7f9bc806ef60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.306+0000 7f4112ffd640 1 -- 172.21.15.40:0/4099747399 <== osd.6 v2:172.21.15.62:6800/25136 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f410406f570 con 0x7f410406caa0 2024-04-22T05:50:42.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.310+0000 7f9bda7fc640 1 -- 172.21.15.40:0/2693920703 <== osd.5 v2:172.21.15.62:6824/25137 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f9bc806f610 con 0x7f9bc806cb40 2024-04-22T05:50:42.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.310+0000 7f05ccef8640 1 -- 172.21.15.40:0/3436946160 --> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f05c8067380 con 0x7f05b406cac0 2024-04-22T05:50:42.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.310+0000 7f05affff640 1 -- 172.21.15.40:0/3436946160 <== osd.1 v2:172.21.15.40:6800/24984 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f05c8067380 con 0x7f05b406cac0 2024-04-22T05:50:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.310+0000 7f05adffb640 1 -- 172.21.15.40:0/3436946160 >> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] conn(0x7f05b406cac0 msgr2=0x7f05b406eee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.314+0000 7f05adffb640 1 --2- 172.21.15.40:0/3436946160 >> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] conn(0x7f05b406cac0 0x7f05b406eee0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.314+0000 7f05adffb640 1 -- 172.21.15.40:0/3436946160 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f05b40666e0 msgr2=0x7f05b4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.314+0000 7f05adffb640 1 --2- 172.21.15.40:0/3436946160 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f05b40666e0 0x7f05b4068ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f05c00af000 tx=0x7f05c00a7070 comp rx=0 tx=0).stop 2024-04-22T05:50:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.314+0000 7f05adffb640 1 -- 172.21.15.40:0/3436946160 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f05c81417f0 msgr2=0x7f05c80abc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.314+0000 7f05adffb640 1 --2- 172.21.15.40:0/3436946160 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f05c81417f0 0x7f05c80abc60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f05b800e7f0 tx=0x7f05b800ecb0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.314+0000 7f05c7fff640 1 -- 172.21.15.40:0/3436946160 reap_dead start 2024-04-22T05:50:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.314+0000 7f05adffb640 1 -- 172.21.15.40:0/3436946160 shutdown_connections 2024-04-22T05:50:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.314+0000 7f05adffb640 1 -- 172.21.15.40:0/3436946160 >> 172.21.15.40:0/3436946160 conn(0x7f05c80b27a0 msgr2=0x7f05c813a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.314+0000 7f05adffb640 1 -- 172.21.15.40:0/3436946160 shutdown_connections 2024-04-22T05:50:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.314+0000 7f05adffb640 1 -- 172.21.15.40:0/3436946160 wait complete. 2024-04-22T05:50:42.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.318+0000 7fe2d4d80640 1 -- 172.21.15.40:0/2210182230 --> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe2b0002b60 con 0x7fe2bc06caa0 2024-04-22T05:50:42.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.318+0000 7fe2b7fff640 1 -- 172.21.15.40:0/2210182230 <== osd.0 v2:172.21.15.40:6816/24983 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe2b0002b60 con 0x7fe2bc06caa0 2024-04-22T05:50:42.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f2a1689c640 1 -- 172.21.15.40:0/2448623042 --> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2a0802fef0 con 0x7f29f406cb70 2024-04-22T05:50:42.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f01de225640 1 -- 172.21.15.40:0/2846142691 --> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f01b8002b60 con 0x7f01bc06ca70 2024-04-22T05:50:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f2a067fc640 1 -- 172.21.15.40:0/2448623042 <== osd.4 v2:172.21.15.62:6816/25135 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2a0802fef0 con 0x7f29f406cb70 2024-04-22T05:50:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f760aec9640 1 -- 172.21.15.40:0/2543326422 --> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7604067570 con 0x7f76041f9950 2024-04-22T05:50:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f2a1689c640 1 -- 172.21.15.40:0/2448623042 >> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] conn(0x7f29f406cb70 msgr2=0x7f29f406ef90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f2a1689c640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] conn(0x7f29f406cb70 0x7f29f406ef90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f2a1689c640 1 -- 172.21.15.40:0/2448623042 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f29f4066790 msgr2=0x7f29f4068c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f2a1689c640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f29f4066790 0x7f29f4068c50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c000fe0 tx=0x7f2a0c0015a0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f2a1689c640 1 -- 172.21.15.40:0/2448623042 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2a080f5ab0 msgr2=0x7f2a0818efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f2a1689c640 1 --2- 172.21.15.40:0/2448623042 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2a080f5ab0 0x7f2a0818efa0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000b3e0 tx=0x7f2a00009f90 comp rx=0 tx=0).stop 2024-04-22T05:50:42.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f2a1609b640 1 -- 172.21.15.40:0/2448623042 reap_dead start 2024-04-22T05:50:42.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f01cdffb640 1 -- 172.21.15.40:0/2846142691 <== osd.7 v2:172.21.15.62:6808/25138 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f01b8002b60 con 0x7f01bc06ca70 2024-04-22T05:50:42.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f75fa7fc640 1 -- 172.21.15.40:0/2543326422 <== osd.3 v2:172.21.15.40:6817/24985 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7604067570 con 0x7f76041f9950 2024-04-22T05:50:42.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f2a1689c640 1 -- 172.21.15.40:0/2448623042 shutdown_connections 2024-04-22T05:50:42.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f2a1689c640 1 -- 172.21.15.40:0/2448623042 >> 172.21.15.40:0/2448623042 conn(0x7f2a08005ec0 msgr2=0x7f2a08004930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f2a1689c640 1 -- 172.21.15.40:0/2448623042 shutdown_connections 2024-04-22T05:50:42.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f2a1689c640 1 -- 172.21.15.40:0/2448623042 wait complete. 2024-04-22T05:50:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f01de225640 1 -- 172.21.15.40:0/2846142691 >> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] conn(0x7f01bc06ca70 msgr2=0x7f01bc06ee90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f01de225640 1 --2- 172.21.15.40:0/2846142691 >> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] conn(0x7f01bc06ca70 0x7f01bc06ee90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f01de225640 1 -- 172.21.15.40:0/2846142691 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f01bc066690 msgr2=0x7f01bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f01de225640 1 --2- 172.21.15.40:0/2846142691 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f01bc066690 0x7f01bc068b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f01d80a89e0 tx=0x7f01c8008040 comp rx=0 tx=0).stop 2024-04-22T05:50:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f01de225640 1 -- 172.21.15.40:0/2846142691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d8073540 msgr2=0x7f01d8145b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f01de225640 1 --2- 172.21.15.40:0/2846142691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d8073540 0x7f01d8145b30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c7f0 tx=0x7f01d000ccb0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f01dda24640 1 -- 172.21.15.40:0/2846142691 reap_dead start 2024-04-22T05:50:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f01de225640 1 -- 172.21.15.40:0/2846142691 shutdown_connections 2024-04-22T05:50:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f01de225640 1 -- 172.21.15.40:0/2846142691 >> 172.21.15.40:0/2846142691 conn(0x7f01d80af950 msgr2=0x7f01d8065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f760aec9640 1 -- 172.21.15.40:0/2543326422 >> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] conn(0x7f76041f9950 msgr2=0x7f76041f9d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f760aec9640 1 --2- 172.21.15.40:0/2543326422 >> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] conn(0x7f76041f9950 0x7f76041f9d30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f01de225640 1 -- 172.21.15.40:0/2846142691 shutdown_connections 2024-04-22T05:50:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f01de225640 1 -- 172.21.15.40:0/2846142691 wait complete. 2024-04-22T05:50:42.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f760aec9640 1 -- 172.21.15.40:0/2543326422 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f75e8066690 msgr2=0x7f75e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f760aec9640 1 --2- 172.21.15.40:0/2543326422 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f75e8066690 0x7f75e8068b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f760415e8a0 tx=0x7f75f4007b60 comp rx=0 tx=0).stop 2024-04-22T05:50:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f760aec9640 1 -- 172.21.15.40:0/2543326422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760406e480 msgr2=0x7f760415e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f760aec9640 1 --2- 172.21.15.40:0/2543326422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760406e480 0x7f760415e360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75fc019fb0 tx=0x7f75fc019120 comp rx=0 tx=0).stop 2024-04-22T05:50:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f760a6c8640 1 -- 172.21.15.40:0/2543326422 reap_dead start 2024-04-22T05:50:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f760aec9640 1 -- 172.21.15.40:0/2543326422 shutdown_connections 2024-04-22T05:50:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.322+0000 7f760aec9640 1 -- 172.21.15.40:0/2543326422 >> 172.21.15.40:0/2543326422 conn(0x7f76040b29e0 msgr2=0x7f76040b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.326+0000 7f760aec9640 1 -- 172.21.15.40:0/2543326422 shutdown_connections 2024-04-22T05:50:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.326+0000 7f760aec9640 1 -- 172.21.15.40:0/2543326422 wait complete. 2024-04-22T05:50:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.326+0000 7fe2b5ffb640 1 -- 172.21.15.40:0/2210182230 >> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] conn(0x7fe2bc06caa0 msgr2=0x7fe2bc06eec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.326+0000 7fe2b5ffb640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] conn(0x7fe2bc06caa0 0x7fe2bc06eec0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.326+0000 7fe2b5ffb640 1 -- 172.21.15.40:0/2210182230 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe2bc0666c0 msgr2=0x7fe2bc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.326+0000 7fe2b5ffb640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe2bc0666c0 0x7fe2bc068b80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40ae000 tx=0x7fe2c40b5040 comp rx=0 tx=0).stop 2024-04-22T05:50:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.326+0000 7fe2b5ffb640 1 -- 172.21.15.40:0/2210182230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2d00709e0 msgr2=0x7fe2d00aa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.326+0000 7fe2b5ffb640 1 --2- 172.21.15.40:0/2210182230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2d00709e0 0x7fe2d00aa590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000b340 tx=0x7fe2c0009120 comp rx=0 tx=0).stop 2024-04-22T05:50:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.326+0000 7fe2cffff640 1 -- 172.21.15.40:0/2210182230 reap_dead start 2024-04-22T05:50:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.326+0000 7fe2b5ffb640 1 -- 172.21.15.40:0/2210182230 shutdown_connections 2024-04-22T05:50:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.326+0000 7fe2b5ffb640 1 -- 172.21.15.40:0/2210182230 >> 172.21.15.40:0/2210182230 conn(0x7fe2d0073af0 msgr2=0x7fe2d014e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.326+0000 7fe2b5ffb640 1 -- 172.21.15.40:0/2210182230 shutdown_connections 2024-04-22T05:50:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.326+0000 7fe2b5ffb640 1 -- 172.21.15.40:0/2210182230 wait complete. 2024-04-22T05:50:42.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7ffa27fff640 1 -- 172.21.15.40:0/3001873620 --> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa200077a0 con 0x7ffa1406c950 2024-04-22T05:50:42.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7ffa0f7fe640 1 -- 172.21.15.40:0/3001873620 <== osd.2 v2:172.21.15.40:6808/24986 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffa200077a0 con 0x7ffa1406c950 2024-04-22T05:50:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7ffa27fff640 1 -- 172.21.15.40:0/3001873620 >> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] conn(0x7ffa1406c950 msgr2=0x7ffa1406ed70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7ffa27fff640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] conn(0x7ffa1406c950 0x7ffa1406ed70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7ffa27fff640 1 -- 172.21.15.40:0/3001873620 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ffa140666e0 msgr2=0x7ffa14068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7ffa27fff640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ffa140666e0 0x7ffa14068ba0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffa1801aaa0 tx=0x7ffa180a6000 comp rx=0 tx=0).stop 2024-04-22T05:50:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7ffa27fff640 1 -- 172.21.15.40:0/3001873620 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ffa200375e0 msgr2=0x7ffa201671b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7ffa27fff640 1 --2- 172.21.15.40:0/3001873620 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ffa200375e0 0x7ffa201671b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffa28063b90 tx=0x7ffa28067b00 comp rx=0 tx=0).stop 2024-04-22T05:50:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7ffa277fe640 1 -- 172.21.15.40:0/3001873620 reap_dead start 2024-04-22T05:50:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7ffa27fff640 1 -- 172.21.15.40:0/3001873620 shutdown_connections 2024-04-22T05:50:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7ffa27fff640 1 -- 172.21.15.40:0/3001873620 >> 172.21.15.40:0/3001873620 conn(0x7ffa2003a8a0 msgr2=0x7ffa200c73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7f412359e640 1 -- 172.21.15.40:0/4099747399 --> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f412413b040 con 0x7f410406caa0 2024-04-22T05:50:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7ffa27fff640 1 -- 172.21.15.40:0/3001873620 shutdown_connections 2024-04-22T05:50:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.334+0000 7ffa27fff640 1 -- 172.21.15.40:0/3001873620 wait complete. 2024-04-22T05:50:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f9bea974640 1 -- 172.21.15.40:0/2693920703 --> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9bdc0bef40 con 0x7f9bc806cb40 2024-04-22T05:50:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f4112ffd640 1 -- 172.21.15.40:0/4099747399 <== osd.6 v2:172.21.15.62:6800/25136 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f412413b040 con 0x7f410406caa0 2024-04-22T05:50:42.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f9bda7fc640 1 -- 172.21.15.40:0/2693920703 <== osd.5 v2:172.21.15.62:6824/25137 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9bdc0bef40 con 0x7f9bc806cb40 2024-04-22T05:50:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f412359e640 1 -- 172.21.15.40:0/4099747399 >> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] conn(0x7f410406caa0 msgr2=0x7f410406eec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f412359e640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] conn(0x7f410406caa0 0x7f410406eec0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f412359e640 1 -- 172.21.15.40:0/4099747399 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f41040666c0 msgr2=0x7f4104068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f412359e640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f41040666c0 0x7f4104068b80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f411c0ae000 tx=0x7f411c0a7070 comp rx=0 tx=0).stop 2024-04-22T05:50:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f412359e640 1 -- 172.21.15.40:0/4099747399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f412406f3b0 msgr2=0x7f41240a77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f412359e640 1 --2- 172.21.15.40:0/4099747399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f412406f3b0 0x7f41240a77d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f411400e870 tx=0x7f411400ec70 comp rx=0 tx=0).stop 2024-04-22T05:50:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f4122d9d640 1 -- 172.21.15.40:0/4099747399 reap_dead start 2024-04-22T05:50:42.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f412359e640 1 -- 172.21.15.40:0/4099747399 shutdown_connections 2024-04-22T05:50:42.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f412359e640 1 -- 172.21.15.40:0/4099747399 >> 172.21.15.40:0/4099747399 conn(0x7f41240b3980 msgr2=0x7f412406ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f412359e640 1 -- 172.21.15.40:0/4099747399 shutdown_connections 2024-04-22T05:50:42.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.338+0000 7f412359e640 1 -- 172.21.15.40:0/4099747399 wait complete. 2024-04-22T05:50:42.369 INFO:teuthology.orchestra.run.smithi040.stdout:55834574850 2024-04-22T05:50:42.370 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-22T05:50:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.346+0000 7f9bea974640 1 -- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] conn(0x7f9bc806cb40 msgr2=0x7f9bc806ef60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.346+0000 7f9bea974640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] conn(0x7f9bc806cb40 0x7f9bc806ef60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.346+0000 7f9bea974640 1 -- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9bc8066760 msgr2=0x7f9bc8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.346+0000 7f9bea974640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9bc8066760 0x7f9bc8068c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9be00011f0 tx=0x7f9be0009040 comp rx=0 tx=0).stop 2024-04-22T05:50:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.346+0000 7f9bea974640 1 -- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bdc032ee0 msgr2=0x7f9bdc008d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.346+0000 7f9bea974640 1 --2- 172.21.15.40:0/2693920703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bdc032ee0 0x7f9bdc008d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9be4063b90 tx=0x7f9be4067200 comp rx=0 tx=0).stop 2024-04-22T05:50:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.346+0000 7f9bea173640 1 -- 172.21.15.40:0/2693920703 reap_dead start 2024-04-22T05:50:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.346+0000 7f9bea974640 1 -- 172.21.15.40:0/2693920703 shutdown_connections 2024-04-22T05:50:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.346+0000 7f9bea974640 1 -- 172.21.15.40:0/2693920703 >> 172.21.15.40:0/2693920703 conn(0x7f9bdc006270 msgr2=0x7f9bdc0c2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.346+0000 7f9bea974640 1 -- 172.21.15.40:0/2693920703 shutdown_connections 2024-04-22T05:50:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.346+0000 7f9bea974640 1 -- 172.21.15.40:0/2693920703 wait complete. 2024-04-22T05:50:42.372 INFO:teuthology.orchestra.run.smithi040.stdout:60129542148 2024-04-22T05:50:42.373 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-22T05:50:42.373 INFO:teuthology.orchestra.run.smithi040.stdout:60129542147 2024-04-22T05:50:42.373 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-22T05:50:42.374 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-22T05:50:42.374 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-22T05:50:42.375 INFO:teuthology.orchestra.run.smithi040.stdout:55834574851 2024-04-22T05:50:42.375 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-22T05:50:42.376 INFO:teuthology.orchestra.run.smithi040.stdout:60129542146 2024-04-22T05:50:42.376 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-22T05:50:42.377 INFO:teuthology.orchestra.run.smithi040.stdout:60129542148 2024-04-22T05:50:42.377 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-22T05:50:42.378 INFO:teuthology.orchestra.run.smithi040.stdout:60129542147 2024-04-22T05:50:42.379 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-22T05:50:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7ff3b20af640 1 Processor -- start 2024-04-22T05:50:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7ff3b20af640 1 -- start start 2024-04-22T05:50:42.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7f4ab4bb6640 1 Processor -- start 2024-04-22T05:50:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7f4ab4bb6640 1 -- start start 2024-04-22T05:50:42.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7f98873b6640 1 Processor -- start 2024-04-22T05:50:42.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7f4ab4bb6640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4ab006f350 0x7f4ab00ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7f4ab4bb6640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4ab0070ed0 0x7f4ab0071320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7f4ab4bb6640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab006f730 con 0x7f4ab006e9d0 2024-04-22T05:50:42.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7f4ab4bb6640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab0070b90 con 0x7f4ab006f350 2024-04-22T05:50:42.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7f4ab4bb6640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00af010 con 0x7f4ab0070ed0 2024-04-22T05:50:42.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7f4aaed9d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4ab0070ed0 0x7f4ab0071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7f4aaed9d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4ab0070ed0 0x7f4ab0071320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35080/0 (socket says 172.21.15.40:35080) 2024-04-22T05:50:42.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7f4aaed9d640 1 -- 172.21.15.40:0/1094375391 learned_addr learned my addr 172.21.15.40:0/1094375391 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7f4aae59c640 1 --2- 172.21.15.40:0/1094375391 >> v2:172.21.15.62:3300/0 conn(0x7f4ab006f350 0x7f4ab00ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7fe4e3fff640 1 Processor -- start 2024-04-22T05:50:42.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7ff3b20af640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3ac0af6b0 0x7ff3ac06bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7ff3b20af640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3ac0b05a0 0x7ff3ac144570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7ff3b20af640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac06b1d0 con 0x7ff3ac06c0f0 2024-04-22T05:50:42.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7ff3b20af640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ac0ae670 con 0x7ff3ac0b05a0 2024-04-22T05:50:42.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7ff3b20af640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ac06b4b0 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff3b08ac640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3ac0b05a0 0x7ff3ac144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff3b08ac640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3ac0b05a0 0x7ff3ac144570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59562/0 (socket says 172.21.15.40:59562) 2024-04-22T05:50:42.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff3b08ac640 1 -- 172.21.15.40:0/2165247037 learned_addr learned my addr 172.21.15.40:0/2165247037 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff3b10ad640 1 --2- 172.21.15.40:0/2165247037 >> v2:172.21.15.62:3300/0 conn(0x7ff3ac0af6b0 0x7ff3ac06bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe907cca640 1 Processor -- start 2024-04-22T05:50:42.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.550+0000 7f98873b6640 1 -- start start 2024-04-22T05:50:42.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f98873b6640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f98800716b0 0x7f9880071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f98873b6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f988006fcf0 0x7f98801402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f98873b6640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9880071a90 con 0x7f9880140800 2024-04-22T05:50:42.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f98873b6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880070cc0 con 0x7f988006fcf0 2024-04-22T05:50:42.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f98873b6640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988006ed70 con 0x7f98800716b0 2024-04-22T05:50:42.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f9884951640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f988006fcf0 0x7f98801402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f9885152640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f98800716b0 0x7f9880071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f9884951640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f988006fcf0 0x7f98801402c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59566/0 (socket says 172.21.15.40:59566) 2024-04-22T05:50:42.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f9884951640 1 -- 172.21.15.40:0/1076980408 learned_addr learned my addr 172.21.15.40:0/1076980408 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f4aaed9d640 1 -- 172.21.15.40:0/1094375391 >> v2:172.21.15.62:3300/0 conn(0x7f4ab006f350 msgr2=0x7f4ab00ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f4aaed9d640 1 --2- 172.21.15.40:0/1094375391 >> v2:172.21.15.62:3300/0 conn(0x7f4ab006f350 0x7f4ab00ac5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f4aaed9d640 1 -- 172.21.15.40:0/1094375391 >> v1:172.21.15.62:6789/0 conn(0x7f4ab006e9d0 legacy=0x7f4ab0142590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f4aaed9d640 1 -- 172.21.15.40:0/1094375391 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab006e4b0 con 0x7f4ab0070ed0 2024-04-22T05:50:42.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe916bd1640 1 Processor -- start 2024-04-22T05:50:42.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f9884951640 1 -- 172.21.15.40:0/1076980408 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f98800716b0 msgr2=0x7f9880071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f9884951640 1 --2- 172.21.15.40:0/1076980408 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f98800716b0 0x7f9880071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f9884951640 1 -- 172.21.15.40:0/1076980408 >> v1:172.21.15.62:6789/0 conn(0x7f9880140800 legacy=0x7f98800ac6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:42.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f9884951640 1 -- 172.21.15.40:0/1076980408 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988006fa30 con 0x7f988006fcf0 2024-04-22T05:50:42.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e3fff640 1 -- start start 2024-04-22T05:50:42.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e3fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4dc0c1f80 0x7fe4dc0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e3fff640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4dc0378e0 con 0x7fe4dc02c4f0 2024-04-22T05:50:42.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e3fff640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4dc02d830 con 0x7fe4dc037500 2024-04-22T05:50:42.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e3fff640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc02c0a0 con 0x7fe4dc0c1f80 2024-04-22T05:50:42.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e2ffd640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe4dc037500 0x7fe4dc0c1870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:58388/0 (socket says 172.21.15.40:58388) 2024-04-22T05:50:42.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e2ffd640 1 -- 172.21.15.40:0/80790943 learned_addr learned my addr 172.21.15.40:0/80790943 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:50:42.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe907cca640 1 -- start start 2024-04-22T05:50:42.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff3b08ac640 1 -- 172.21.15.40:0/2165247037 >> v2:172.21.15.62:3300/0 conn(0x7ff3ac0af6b0 msgr2=0x7ff3ac06bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff3b08ac640 1 --2- 172.21.15.40:0/2165247037 >> v2:172.21.15.62:3300/0 conn(0x7ff3ac0af6b0 0x7ff3ac06bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff3b08ac640 1 -- 172.21.15.40:0/2165247037 >> v1:172.21.15.62:6789/0 conn(0x7ff3ac06c0f0 legacy=0x7ff3ac148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff3b08ac640 1 -- 172.21.15.40:0/2165247037 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac06b790 con 0x7ff3ac0b05a0 2024-04-22T05:50:42.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590362063 0 0) 0x7fe4dc02d830 con 0x7fe4dc037500 2024-04-22T05:50:42.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe916bd1640 1 -- start start 2024-04-22T05:50:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe916bd1640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe9080c99b0 0x7fe9080cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe916bd1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe908034fd0 con 0x7fe908034bf0 2024-04-22T05:50:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe916bd1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe908032100 con 0x7fe90802fc10 2024-04-22T05:50:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe916bd1640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe908033190 con 0x7fe9080c99b0 2024-04-22T05:50:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4cc003530 con 0x7fe4dc037500 2024-04-22T05:50:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe9153ce640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe9080c99b0 0x7fe9080cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe9153ce640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe9080c99b0 0x7fe9080cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35110/0 (socket says 172.21.15.40:35110) 2024-04-22T05:50:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe9153ce640 1 -- 172.21.15.40:0/1814306976 learned_addr learned my addr 172.21.15.40:0/1814306976 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe9153ce640 1 -- 172.21.15.40:0/1814306976 >> v1:172.21.15.62:6789/0 conn(0x7fe90802fc10 legacy=0x7fe9080cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe9153ce640 1 -- 172.21.15.40:0/1814306976 >> v1:172.21.15.40:6789/0 conn(0x7fe908034bf0 legacy=0x7fe9080c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 141441169 0 0) 0x7fe4cc003530 con 0x7fe4dc037500 2024-04-22T05:50:42.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4dc0c1f80 msgr2=0x7fe4dc0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e27fc640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4dc0c1f80 0x7fe4dc0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4dc0c1f80 0x7fe4dc0c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 >> v1:172.21.15.40:6789/0 conn(0x7fe4dc02c4f0 legacy=0x7fe4dc0cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe907cca640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe9000afc10 0x7fe90006f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc02b400 con 0x7fe4dc037500 2024-04-22T05:50:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe907cca640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe90006eed0 0x7fe900071320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 <== mon.1 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268782488 0 0) 0x7fe4e404e690 con 0x7fe4dc037500 2024-04-22T05:50:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 >> v1:172.21.15.62:6789/0 conn(0x7fe4dc037500 legacy=0x7fe4dc0c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe905a66640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe9000afc10 0x7fe90006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe905a66640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe9000afc10 0x7fe90006f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59570/0 (socket says 172.21.15.40:59570) 2024-04-22T05:50:42.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe905a66640 1 -- 172.21.15.40:0/3100540775 learned_addr learned my addr 172.21.15.40:0/3100540775 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4cc003980 0x7fe4cc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4cc0047d0 0x7fe4cc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f5b11d74640 1 Processor -- start 2024-04-22T05:50:42.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc005190 0x7fe4cc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc02c0a0 con 0x7fe4cc005190 2024-04-22T05:50:42.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc00e2a0 con 0x7fe4cc003980 2024-04-22T05:50:42.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc00e460 con 0x7fe4cc0047d0 2024-04-22T05:50:42.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e27fc640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4dc0c1f80 0x7fe4dc0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:50:42.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe907cca640 1 -- 172.21.15.40:0/3100540775 --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9000afff0 con 0x7fe900071860 2024-04-22T05:50:42.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e2ffd640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4cc003980 0x7fe4cc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe907cca640 1 -- 172.21.15.40:0/3100540775 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90006e760 con 0x7fe9000afc10 2024-04-22T05:50:42.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e27fc640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4cc0047d0 0x7fe4cc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe905265640 1 --2- 172.21.15.40:0/3100540775 >> v2:172.21.15.62:3300/0 conn(0x7fe90006eed0 0x7fe900071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe907cca640 1 -- 172.21.15.40:0/3100540775 --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90006feb0 con 0x7fe90006eed0 2024-04-22T05:50:42.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe9153ce640 1 -- 172.21.15.40:0/1814306976 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9080ccbf0 con 0x7fe9080c99b0 2024-04-22T05:50:42.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7f4aaed9d640 1 --2- 172.21.15.40:0/1094375391 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4ab0070ed0 0x7f4ab0071320 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009870 tx=0x7f4aa801ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d368c911bb3d8a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff3b08ac640 1 --2- 172.21.15.40:0/2165247037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac0b05a0 0x7ff3ac144570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac06d610 tx=0x7ff3a801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2879249130bb71ba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7fe4e37fe640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc005190 0x7fe4cc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe904a64640 1 -- 172.21.15.40:0/3100540775 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949489497 0 0) 0x7fe9000afff0 con 0x7fe900071860 2024-04-22T05:50:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe904a64640 1 -- 172.21.15.40:0/3100540775 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8ec005740 con 0x7fe900071860 2024-04-22T05:50:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe4e2ffd640 1 -- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4cc0047d0 msgr2=0x7fe4cc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f4aad59a640 1 -- 172.21.15.40:0/1094375391 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80a8a30 con 0x7f4ab0070ed0 2024-04-22T05:50:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f4aad59a640 1 -- 172.21.15.40:0/1094375391 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa80a8b90 con 0x7f4ab0070ed0 2024-04-22T05:50:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f4aad59a640 1 -- 172.21.15.40:0/1094375391 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80b0440 con 0x7f4ab0070ed0 2024-04-22T05:50:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe905a66640 1 -- 172.21.15.40:0/3100540775 >> v2:172.21.15.62:3300/0 conn(0x7fe90006eed0 msgr2=0x7fe900071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe905a66640 1 --2- 172.21.15.40:0/3100540775 >> v2:172.21.15.62:3300/0 conn(0x7fe90006eed0 0x7fe900071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe905a66640 1 -- 172.21.15.40:0/3100540775 >> v1:172.21.15.62:6789/0 conn(0x7fe900071860 legacy=0x7fe900139ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe905a66640 1 -- 172.21.15.40:0/3100540775 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe900070f00 con 0x7fe9000afc10 2024-04-22T05:50:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff937fff640 1 Processor -- start 2024-04-22T05:50:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b11d74640 1 -- start start 2024-04-22T05:50:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff937fff640 1 -- start start 2024-04-22T05:50:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff937fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff930006de0 0x7ff93002f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff937fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff930032130 0x7ff930031250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff937fff640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9300071c0 con 0x7ff9300cf4d0 2024-04-22T05:50:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff937fff640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff930030d70 con 0x7ff930032130 2024-04-22T05:50:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.554+0000 7ff937fff640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff930033190 con 0x7ff930006de0 2024-04-22T05:50:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff936ffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff930006de0 0x7ff93002f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff936ffd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff930006de0 0x7ff93002f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35138/0 (socket says 172.21.15.40:35138) 2024-04-22T05:50:42.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff936ffd640 1 -- 172.21.15.40:0/2840218319 learned_addr learned my addr 172.21.15.40:0/2840218319 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff936ffd640 1 -- 172.21.15.40:0/2840218319 >> v2:172.21.15.40:3300/0 conn(0x7ff930032130 msgr2=0x7ff930031250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:42.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff936ffd640 1 --2- 172.21.15.40:0/2840218319 >> v2:172.21.15.40:3300/0 conn(0x7ff930032130 0x7ff930031250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe4e2ffd640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4cc0047d0 0x7fe4cc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe4e2ffd640 1 -- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc005190 msgr2=0x7fe4cc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe4e2ffd640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc005190 0x7fe4cc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff936ffd640 1 -- 172.21.15.40:0/2840218319 >> v1:172.21.15.62:6789/0 conn(0x7ff9300cf4d0 legacy=0x7ff9300c9210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff936ffd640 1 -- 172.21.15.40:0/2840218319 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9300cf960 con 0x7ff930006de0 2024-04-22T05:50:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe4e2ffd640 1 -- 172.21.15.40:0/80790943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc003530 con 0x7fe4cc003980 2024-04-22T05:50:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe4e37fe640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc005190 0x7fe4cc00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T05:50:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe4e27fc640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4cc0047d0 0x7fe4cc004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:50:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b11d74640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b0c06f3c0 0x7f5b0c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b11d74640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5b0c0acb80 0x7f5b0c070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b11d74640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b0c0713f0 0x7f5b0c142500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b11d74640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b0c0704d0 con 0x7f5b0c0713f0 2024-04-22T05:50:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b11d74640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b0c0707b0 con 0x7f5b0c0acb80 2024-04-22T05:50:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b11d74640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c06eb00 con 0x7f5b0c06f3c0 2024-04-22T05:50:42.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b0b7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b0c06f3c0 0x7f5b0c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/1094375391 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4ab0070ed0 msgr2=0x7f4ab0071320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f4ab4bb6640 1 --2- 172.21.15.40:0/1094375391 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4ab0070ed0 0x7f4ab0071320 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009870 tx=0x7f4aa801ed30 comp rx=0 tx=0).stop 2024-04-22T05:50:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b0b7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b0c06f3c0 0x7f5b0c06e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35146/0 (socket says 172.21.15.40:35146) 2024-04-22T05:50:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b0b7fe640 1 -- 172.21.15.40:0/3284796339 learned_addr learned my addr 172.21.15.40:0/3284796339 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff3a3fff640 1 -- 172.21.15.40:0/2165247037 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80ab660 con 0x7ff3ac0b05a0 2024-04-22T05:50:42.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff3a3fff640 1 -- 172.21.15.40:0/2165247037 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a80ab7c0 con 0x7ff3ac0b05a0 2024-04-22T05:50:42.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff3a3fff640 1 -- 172.21.15.40:0/2165247037 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80abb50 con 0x7ff3ac0b05a0 2024-04-22T05:50:42.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b0affd640 1 --2- 172.21.15.40:0/3284796339 >> v2:172.21.15.62:3300/0 conn(0x7f5b0c0acb80 0x7f5b0c070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b0b7fe640 1 -- 172.21.15.40:0/3284796339 >> v2:172.21.15.62:3300/0 conn(0x7f5b0c0acb80 msgr2=0x7f5b0c070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/1094375391 shutdown_connections 2024-04-22T05:50:42.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f4ab4bb6640 1 --2- 172.21.15.40:0/1094375391 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4ab0070ed0 0x7f4ab0071320 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f4ab4bb6640 1 --2- 172.21.15.40:0/1094375391 >> v2:172.21.15.62:3300/0 conn(0x7f4ab006f350 0x7f4ab00ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/1094375391 >> 172.21.15.40:0/1094375391 conn(0x7f4ab00b27a0 msgr2=0x7f4ab00b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b0b7fe640 1 --2- 172.21.15.40:0/3284796339 >> v2:172.21.15.62:3300/0 conn(0x7f5b0c0acb80 0x7f5b0c070eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b0b7fe640 1 -- 172.21.15.40:0/3284796339 >> v2:172.21.15.40:3300/0 conn(0x7f5b0c0713f0 msgr2=0x7f5b0c142500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b0b7fe640 1 --2- 172.21.15.40:0/3284796339 >> v2:172.21.15.40:3300/0 conn(0x7f5b0c0713f0 0x7f5b0c142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f5b0b7fe640 1 -- 172.21.15.40:0/3284796339 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c06f7a0 con 0x7f5b0c06f3c0 2024-04-22T05:50:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/1094375391 shutdown_connections 2024-04-22T05:50:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff3b20af640 1 -- 172.21.15.40:0/2165247037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac0b05a0 msgr2=0x7ff3ac144570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff3b20af640 1 --2- 172.21.15.40:0/2165247037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac0b05a0 0x7ff3ac144570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac06d610 tx=0x7ff3a801c440 comp rx=0 tx=0).stop 2024-04-22T05:50:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/1094375391 wait complete. 2024-04-22T05:50:42.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f9884951640 1 --2- 172.21.15.40:0/1076980408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f988006fcf0 0x7f98801402c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f987800ce20 tx=0x7f987801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64ce54e0b970e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff3b20af640 1 -- 172.21.15.40:0/2165247037 shutdown_connections 2024-04-22T05:50:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff3b20af640 1 --2- 172.21.15.40:0/2165247037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac0b05a0 0x7ff3ac144570 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff3b20af640 1 --2- 172.21.15.40:0/2165247037 >> v2:172.21.15.62:3300/0 conn(0x7ff3ac0af6b0 0x7ff3ac06bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff3b20af640 1 -- 172.21.15.40:0/2165247037 >> 172.21.15.40:0/2165247037 conn(0x7ff3ac0b49a0 msgr2=0x7ff3ac0b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe9153ce640 1 --2- 172.21.15.40:0/1814306976 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe9080c99b0 0x7fe9080cbfa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe910072930 tx=0x7fe910095e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb4e1ab7bc6bc6b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe4e2ffd640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4cc003980 0x7fe4cc003da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4067510 tx=0x7fe4e4096390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=facf54c79854ad38 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f9877fff640 1 -- 172.21.15.40:0/1076980408 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987800c2d0 con 0x7f988006fcf0 2024-04-22T05:50:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f9877fff640 1 -- 172.21.15.40:0/1076980408 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987800c430 con 0x7f988006fcf0 2024-04-22T05:50:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f9877fff640 1 -- 172.21.15.40:0/1076980408 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98780a5990 con 0x7f988006fcf0 2024-04-22T05:50:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff3b20af640 1 -- 172.21.15.40:0/2165247037 shutdown_connections 2024-04-22T05:50:42.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f4ab4bb6640 1 Processor -- start 2024-04-22T05:50:42.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe914bcd640 1 -- 172.21.15.40:0/1814306976 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91011eb70 con 0x7fe9080c99b0 2024-04-22T05:50:42.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4125640 con 0x7fe4cc003980 2024-04-22T05:50:42.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe914bcd640 1 -- 172.21.15.40:0/1814306976 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91011ed10 con 0x7fe9080c99b0 2024-04-22T05:50:42.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe914bcd640 1 -- 172.21.15.40:0/1814306976 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910126640 con 0x7fe9080c99b0 2024-04-22T05:50:42.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe905a66640 1 --2- 172.21.15.40:0/3100540775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9000afc10 0x7fe90006f970 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe9000b0360 tx=0x7fe8f001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14403d8aba43cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7f4ab4bb6640 1 -- start start 2024-04-22T05:50:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 -- 172.21.15.40:0/1076980408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f988006fcf0 msgr2=0x7f98801402c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 --2- 172.21.15.40:0/1076980408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f988006fcf0 0x7f98801402c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f987800ce20 tx=0x7f987801c8f0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 -- 172.21.15.40:0/1076980408 shutdown_connections 2024-04-22T05:50:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 --2- 172.21.15.40:0/1076980408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f988006fcf0 0x7f98801402c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 --2- 172.21.15.40:0/1076980408 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f98800716b0 0x7f9880071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 -- 172.21.15.40:0/1076980408 >> 172.21.15.40:0/1076980408 conn(0x7f98800b27a0 msgr2=0x7f98800b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f5b0b7fe640 1 --2- 172.21.15.40:0/3284796339 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b0c06f3c0 0x7f5b0c06e3f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0ae980 tx=0x7f5b0001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a49d3b2c8abd9668 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4ab4bb6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ab006e9d0 0x7f4ab01dd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4ab4bb6640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4ab006f350 0x7f4ab01dd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4ab4bb6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab0070ed0 0x7f4ab01d9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4ab4bb6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab01434b0 con 0x7f4ab0070ed0 2024-04-22T05:50:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4ab4bb6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab0143910 con 0x7f4ab006e9d0 2024-04-22T05:50:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4ab4bb6640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab01ddcf0 con 0x7f4ab006f350 2024-04-22T05:50:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 -- 172.21.15.40:0/1076980408 shutdown_connections 2024-04-22T05:50:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 -- 172.21.15.40:0/1076980408 wait complete. 2024-04-22T05:50:42.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe904a64640 1 -- 172.21.15.40:0/3100540775 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0008be0 con 0x7fe9000afc10 2024-04-22T05:50:42.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f5b0a7fc640 1 -- 172.21.15.40:0/3284796339 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b000ac600 con 0x7f5b0c06f3c0 2024-04-22T05:50:42.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe904a64640 1 -- 172.21.15.40:0/3100540775 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f0008d40 con 0x7fe9000afc10 2024-04-22T05:50:42.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7ff3b20af640 1 -- 172.21.15.40:0/2165247037 wait complete. 2024-04-22T05:50:42.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff3b20af640 1 Processor -- start 2024-04-22T05:50:42.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe904a64640 1 -- 172.21.15.40:0/3100540775 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f000a2f0 con 0x7fe9000afc10 2024-04-22T05:50:42.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f5b0a7fc640 1 -- 172.21.15.40:0/3284796339 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b000a8050 con 0x7f5b0c06f3c0 2024-04-22T05:50:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4aaed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab0070ed0 0x7f4ab01d9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4aaed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab0070ed0 0x7f4ab01d9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59612/0 (socket says 172.21.15.40:59612) 2024-04-22T05:50:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4aaed9d640 1 -- 172.21.15.40:0/875442815 learned_addr learned my addr 172.21.15.40:0/875442815 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4aae59c640 1 --2- 172.21.15.40:0/875442815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ab006e9d0 0x7f4ab01dd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4aae59c640 1 -- 172.21.15.40:0/875442815 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4ab006f350 msgr2=0x7f4ab01dd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4aae59c640 1 --2- 172.21.15.40:0/875442815 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4ab006f350 0x7f4ab01dd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4aae59c640 1 -- 172.21.15.40:0/875442815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab0070ed0 msgr2=0x7f4ab01d9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4aae59c640 1 --2- 172.21.15.40:0/875442815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab0070ed0 0x7f4ab01d9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4aae59c640 1 -- 172.21.15.40:0/875442815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa80094d0 con 0x7f4ab006e9d0 2024-04-22T05:50:42.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe916bd1640 1 -- 172.21.15.40:0/1814306976 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe9080c99b0 msgr2=0x7fe9080cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe916bd1640 1 --2- 172.21.15.40:0/1814306976 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe9080c99b0 0x7fe9080cbfa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe910072930 tx=0x7fe910095e00 comp rx=0 tx=0).stop 2024-04-22T05:50:42.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e4121050 con 0x7fe4cc003980 2024-04-22T05:50:42.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.558+0000 7fe4e1ffb640 1 -- 172.21.15.40:0/80790943 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4125a40 con 0x7fe4cc003980 2024-04-22T05:50:42.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe4e3fff640 1 -- 172.21.15.40:0/80790943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4cc003980 msgr2=0x7fe4cc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe4e3fff640 1 --2- 172.21.15.40:0/80790943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4cc003980 0x7fe4cc003da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4067510 tx=0x7fe4e4096390 comp rx=0 tx=0).stop 2024-04-22T05:50:42.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f5b11d74640 1 -- 172.21.15.40:0/3284796339 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b0c06f3c0 msgr2=0x7f5b0c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f5b11d74640 1 --2- 172.21.15.40:0/3284796339 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b0c06f3c0 0x7f5b0c06e3f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0ae980 tx=0x7f5b0001c4d0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe4e37fe640 1 -- 172.21.15.40:0/80790943 reap_dead start 2024-04-22T05:50:42.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe4e3fff640 1 -- 172.21.15.40:0/80790943 shutdown_connections 2024-04-22T05:50:42.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe4e3fff640 1 -- 172.21.15.40:0/80790943 >> 172.21.15.40:0/80790943 conn(0x7fe4dc03a850 msgr2=0x7fe4dc036b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe907cca640 1 -- 172.21.15.40:0/3100540775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9000afc10 msgr2=0x7fe90006f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe907cca640 1 --2- 172.21.15.40:0/3100540775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9000afc10 0x7fe90006f970 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe9000b0360 tx=0x7fe8f001c3d0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 Processor -- start 2024-04-22T05:50:42.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe4e3fff640 1 -- 172.21.15.40:0/80790943 shutdown_connections 2024-04-22T05:50:42.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f5b11d74640 1 -- 172.21.15.40:0/3284796339 shutdown_connections 2024-04-22T05:50:42.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f5b11d74640 1 --2- 172.21.15.40:0/3284796339 >> v2:172.21.15.62:3300/0 conn(0x7f5b0c0acb80 0x7f5b0c070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f5b11d74640 1 --2- 172.21.15.40:0/3284796339 >> v2:172.21.15.40:3300/0 conn(0x7f5b0c0713f0 0x7f5b0c142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f5b11d74640 1 --2- 172.21.15.40:0/3284796339 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b0c06f3c0 0x7f5b0c06e3f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f5b11d74640 1 -- 172.21.15.40:0/3284796339 >> 172.21.15.40:0/3284796339 conn(0x7f5b0c0b27a0 msgr2=0x7f5b0c0b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe907cca640 1 -- 172.21.15.40:0/3100540775 shutdown_connections 2024-04-22T05:50:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe907cca640 1 --2- 172.21.15.40:0/3100540775 >> v2:172.21.15.62:3300/0 conn(0x7fe90006eed0 0x7fe900071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe907cca640 1 --2- 172.21.15.40:0/3100540775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9000afc10 0x7fe90006f970 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4aae59c640 1 --2- 172.21.15.40:0/875442815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ab006e9d0 0x7f4ab01dd270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009db0 tx=0x7f4aa000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe907cca640 1 -- 172.21.15.40:0/3100540775 >> 172.21.15.40:0/3100540775 conn(0x7fe9000b2da0 msgr2=0x7fe9000af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe907cca640 1 -- 172.21.15.40:0/3100540775 shutdown_connections 2024-04-22T05:50:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe907cca640 1 -- 172.21.15.40:0/3100540775 wait complete. 2024-04-22T05:50:42.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4a9effd640 1 -- 172.21.15.40:0/875442815 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0014480 con 0x7f4ab006e9d0 2024-04-22T05:50:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff3b20af640 1 -- start start 2024-04-22T05:50:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff3b20af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac06c0f0 0x7ff3ac147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff3b20af640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff3ac0af6b0 0x7ff3ac1416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/875442815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab013fdf0 con 0x7f4ab006e9d0 2024-04-22T05:50:42.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/875442815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab0143170 con 0x7f4ab006e9d0 2024-04-22T05:50:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff3b10ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac06c0f0 0x7ff3ac147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff3b08ac640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff3ac0af6b0 0x7ff3ac1416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff3b10ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac06c0f0 0x7ff3ac147e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59628/0 (socket says 172.21.15.40:59628) 2024-04-22T05:50:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff3b10ad640 1 -- 172.21.15.40:0/957244115 learned_addr learned my addr 172.21.15.40:0/957244115 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff3b20af640 1 --2- 172.21.15.40:0/957244115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac0b05a0 0x7ff3ac141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff3b20af640 1 -- 172.21.15.40:0/957244115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac143180 con 0x7ff3ac06c0f0 2024-04-22T05:50:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff3b20af640 1 -- 172.21.15.40:0/957244115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac1379f0 con 0x7ff3ac0b05a0 2024-04-22T05:50:42.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff3b20af640 1 -- 172.21.15.40:0/957244115 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac137cd0 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe907cca640 1 Processor -- start 2024-04-22T05:50:42.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe907cca640 1 -- start start 2024-04-22T05:50:42.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 -- start start 2024-04-22T05:50:42.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f988006fcf0 0x7f9880131a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98800716b0 0x7f98801ddfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880140800 0x7f9880130910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98801dd850 con 0x7f9880140800 2024-04-22T05:50:42.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98801ddaa0 con 0x7f98800716b0 2024-04-22T05:50:42.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f98873b6640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988013abf0 con 0x7f988006fcf0 2024-04-22T05:50:42.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f9885953640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880140800 0x7f9880130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f9885953640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880140800 0x7f9880130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59636/0 (socket says 172.21.15.40:59636) 2024-04-22T05:50:42.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f9885953640 1 -- 172.21.15.40:0/3973915296 learned_addr learned my addr 172.21.15.40:0/3973915296 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f9885152640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f988006fcf0 0x7f9880131a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f9884951640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98800716b0 0x7f98801ddfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f9885152640 1 -- 172.21.15.40:0/3973915296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98800716b0 msgr2=0x7f98801ddfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f9885152640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98800716b0 0x7f98801ddfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f9885152640 1 -- 172.21.15.40:0/3973915296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880140800 msgr2=0x7f9880130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f9885152640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880140800 0x7f9880130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f9885152640 1 -- 172.21.15.40:0/3973915296 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987800ca80 con 0x7f988006fcf0 2024-04-22T05:50:42.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe4e3fff640 1 -- 172.21.15.40:0/80790943 wait complete. 2024-04-22T05:50:42.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7fe4e3fff640 1 Processor -- start 2024-04-22T05:50:42.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe916bd1640 1 -- 172.21.15.40:0/1814306976 shutdown_connections 2024-04-22T05:50:42.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe916bd1640 1 --2- 172.21.15.40:0/1814306976 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe9080c99b0 0x7fe9080cbfa0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe916bd1640 1 -- 172.21.15.40:0/1814306976 >> 172.21.15.40:0/1814306976 conn(0x7fe908006090 msgr2=0x7fe9080064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7f4a9effd640 1 -- 172.21.15.40:0/875442815 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa000f040 con 0x7f4ab006e9d0 2024-04-22T05:50:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e3fff640 1 -- start start 2024-04-22T05:50:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc003980 0x7fe4dc0c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e3fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4dc035c40 0x7fe4dc0c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4dc160290 0x7fe4dc160720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e3fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc0c4d70 con 0x7fe4cc003980 2024-04-22T05:50:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e3fff640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc160c60 con 0x7fe4dc160290 2024-04-22T05:50:42.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e3fff640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc160dd0 con 0x7fe4dc035c40 2024-04-22T05:50:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe907cca640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe90006eed0 0x7fe9000ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe907cca640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe900071860 0x7fe9001d8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe905265640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe900071860 0x7fe9001d8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe905265640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe900071860 0x7fe9001d8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35190/0 (socket says 172.21.15.40:35190) 2024-04-22T05:50:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe905265640 1 -- 172.21.15.40:0/744734911 learned_addr learned my addr 172.21.15.40:0/744734911 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe907cca640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9000afc10 0x7fe9001469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe905a66640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe90006eed0 0x7fe9000ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7f9885152640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f988006fcf0 0x7f9880131a50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f987c00cea0 tx=0x7f987c00cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7f9875ffb640 1 -- 172.21.15.40:0/3973915296 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c00e240 con 0x7f988006fcf0 2024-04-22T05:50:42.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7f9875ffb640 1 -- 172.21.15.40:0/3973915296 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c00e3a0 con 0x7f988006fcf0 2024-04-22T05:50:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7f9875ffb640 1 -- 172.21.15.40:0/3973915296 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c00ae80 con 0x7f988006fcf0 2024-04-22T05:50:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff3b08ac640 1 -- 172.21.15.40:0/957244115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac0b05a0 msgr2=0x7ff3ac141c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff3b08ac640 1 --2- 172.21.15.40:0/957244115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac0b05a0 0x7ff3ac141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff3b08ac640 1 -- 172.21.15.40:0/957244115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac06c0f0 msgr2=0x7ff3ac147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff3b08ac640 1 --2- 172.21.15.40:0/957244115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac06c0f0 0x7ff3ac147e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff3b08ac640 1 -- 172.21.15.40:0/957244115 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a80094d0 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff3b08ac640 1 --2- 172.21.15.40:0/957244115 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff3ac0af6b0 0x7ff3ac1416f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80ac000 tx=0x7ff3a801ac80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc003980 0x7fe4dc0c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc003980 0x7fe4dc0c3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59638/0 (socket says 172.21.15.40:59638) 2024-04-22T05:50:42.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e2ffd640 1 -- 172.21.15.40:0/4208858245 learned_addr learned my addr 172.21.15.40:0/4208858245 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e37fe640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4dc160290 0x7fe4dc160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe906267640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9000afc10 0x7fe9001469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe907cca640 1 -- 172.21.15.40:0/744734911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9001d8150 con 0x7fe9000afc10 2024-04-22T05:50:42.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e37fe640 1 -- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4dc160290 msgr2=0x7fe4dc160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-22T05:50:42.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e37fe640 1 -- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4dc160290 msgr2=0x7fe4dc160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T05:50:42.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e37fe640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4dc160290 0x7fe4dc160720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:42.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e2ffd640 1 -- 172.21.15.40:0/4208858245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc003980 msgr2=0x7fe4dc0c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-22T05:50:42.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e2ffd640 1 -- 172.21.15.40:0/4208858245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc003980 msgr2=0x7fe4dc0c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T05:50:42.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e2ffd640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc003980 0x7fe4dc0c3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:50:42.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e37fe640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4dc160290 0x7fe4dc160720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T05:50:42.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e2ffd640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc003980 0x7fe4dc0c3b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T05:50:42.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe907cca640 1 -- 172.21.15.40:0/744734911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000a9fd0 con 0x7fe90006eed0 2024-04-22T05:50:42.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7f98873b6640 1 -- 172.21.15.40:0/3973915296 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98801de500 con 0x7f988006fcf0 2024-04-22T05:50:42.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7f98873b6640 1 -- 172.21.15.40:0/3973915296 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9880131020 con 0x7f988006fcf0 2024-04-22T05:50:42.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7f4a9effd640 1 -- 172.21.15.40:0/875442815 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0014800 con 0x7f4ab006e9d0 2024-04-22T05:50:42.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe907cca640 1 -- 172.21.15.40:0/744734911 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9001d8360 con 0x7fe900071860 2024-04-22T05:50:42.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7f98873b6640 1 -- 172.21.15.40:0/3973915296 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9880067570 con 0x7f988006fcf0 2024-04-22T05:50:42.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/875442815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a94005050 con 0x7f4ab006e9d0 2024-04-22T05:50:42.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff3a1ffb640 1 -- 172.21.15.40:0/957244115 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80b4800 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff3a1ffb640 1 -- 172.21.15.40:0/957244115 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a80b4960 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff936ffd640 1 --2- 172.21.15.40:0/2840218319 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff930006de0 0x7ff93002f5a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff938072930 tx=0x7ff938095db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce1a836fa4ef70a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff935ffb640 1 -- 172.21.15.40:0/2840218319 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93811eb60 con 0x7ff930006de0 2024-04-22T05:50:42.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff935ffb640 1 -- 172.21.15.40:0/2840218319 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93811ed00 con 0x7ff930006de0 2024-04-22T05:50:42.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.562+0000 7ff935ffb640 1 -- 172.21.15.40:0/2840218319 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938126640 con 0x7ff930006de0 2024-04-22T05:50:42.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff937fff640 1 -- 172.21.15.40:0/2840218319 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff930006de0 msgr2=0x7ff93002f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff937fff640 1 --2- 172.21.15.40:0/2840218319 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff930006de0 0x7ff93002f5a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff938072930 tx=0x7ff938095db0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff937fff640 1 -- 172.21.15.40:0/2840218319 shutdown_connections 2024-04-22T05:50:42.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff937fff640 1 --2- 172.21.15.40:0/2840218319 >> v2:172.21.15.40:3300/0 conn(0x7ff930032130 0x7ff930031250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff937fff640 1 --2- 172.21.15.40:0/2840218319 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff930006de0 0x7ff93002f5a0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff938072930 tx=0x7ff938095db0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff937fff640 1 -- 172.21.15.40:0/2840218319 >> 172.21.15.40:0/2840218319 conn(0x7ff930005ec0 msgr2=0x7ff9300062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff937fff640 1 -- 172.21.15.40:0/2840218319 shutdown_connections 2024-04-22T05:50:42.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff3b20af640 1 -- 172.21.15.40:0/957244115 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac135da0 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe905265640 1 -- 172.21.15.40:0/744734911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe90006eed0 msgr2=0x7fe9000ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe905265640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe90006eed0 0x7fe9000ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe905265640 1 -- 172.21.15.40:0/744734911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9000afc10 msgr2=0x7fe9001469e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe905265640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9000afc10 0x7fe9001469e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe905265640 1 -- 172.21.15.40:0/744734911 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f0008050 con 0x7fe900071860 2024-04-22T05:50:42.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff3b20af640 1 -- 172.21.15.40:0/957244115 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac148560 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe906267640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9000afc10 0x7fe9001469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:50:42.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe905a66640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe90006eed0 0x7fe9000ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:50:42.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe905265640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe900071860 0x7fe9001d8880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00e930 tx=0x7fe8fc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff937fff640 1 -- 172.21.15.40:0/2840218319 wait complete. 2024-04-22T05:50:42.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff937fff640 1 Processor -- start 2024-04-22T05:50:42.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7f4a9effd640 1 -- 172.21.15.40:0/875442815 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4aa0014960 con 0x7f4ab006e9d0 2024-04-22T05:50:42.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7f4a9effd640 1 --2- 172.21.15.40:0/875442815 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4a90066690 0x7f4a90068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe916bd1640 1 -- 172.21.15.40:0/1814306976 shutdown_connections 2024-04-22T05:50:42.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe916bd1640 1 -- 172.21.15.40:0/1814306976 wait complete. 2024-04-22T05:50:42.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff3a1ffb640 1 -- 172.21.15.40:0/957244115 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80b4d10 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe916bd1640 1 Processor -- start 2024-04-22T05:50:42.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7f4a9effd640 1 -- 172.21.15.40:0/875442815 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4aa0084d40 con 0x7f4ab006e9d0 2024-04-22T05:50:42.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe4e27fc640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4dc035c40 0x7fe4dc0c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7ff937fff640 1 -- start start 2024-04-22T05:50:42.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe4e27fc640 1 -- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4dc160290 msgr2=0x7fe4dc160720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T05:50:42.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe4e27fc640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4dc160290 0x7fe4dc160720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe4e27fc640 1 -- 172.21.15.40:0/4208858245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc003980 msgr2=0x7fe4dc0c3b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T05:50:42.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe4e27fc640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc003980 0x7fe4dc0c3b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe4e27fc640 1 -- 172.21.15.40:0/4208858245 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc02baa0 con 0x7fe4dc035c40 2024-04-22T05:50:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff937fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff930032130 0x7ff930037920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe4e27fc640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4dc035c40 0x7fe4dc0c46a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800a4c0 tx=0x7fe4d800fd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7fe916bd1640 1 -- start start 2024-04-22T05:50:42.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe916bd1640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe90802fc10 0x7fe908161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe916bd1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe908034bf0 0x7fe9080c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe916bd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe908160bc0 0x7fe908162490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe916bd1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9081613b0 con 0x7fe908034bf0 2024-04-22T05:50:42.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe916bd1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe908161050 con 0x7fe908160bc0 2024-04-22T05:50:42.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe916bd1640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe908164880 con 0x7fe90802fc10 2024-04-22T05:50:42.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7f4aadd9b640 1 --2- 172.21.15.40:0/875442815 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4a90066690 0x7f4a90068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff937fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9300cf4d0 0x7ff9300b9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff937fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9300380f0 0x7ff9300b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff937fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9300bb230 con 0x7ff9300380f0 2024-04-22T05:50:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff937fff640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff930038580 con 0x7ff930032130 2024-04-22T05:50:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff3977fe640 1 -- 172.21.15.40:0/957244115 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38c005050 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe9153ce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe908034bf0 0x7fe9080c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe9153ce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe908034bf0 0x7fe9080c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59666/0 (socket says 172.21.15.40:59666) 2024-04-22T05:50:42.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe9153ce640 1 -- 172.21.15.40:0/726509246 learned_addr learned my addr 172.21.15.40:0/726509246 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff937fff640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff930037e60 con 0x7ff9300cf4d0 2024-04-22T05:50:42.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.566+0000 7f9875ffb640 1 -- 172.21.15.40:0/3973915296 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f987c015070 con 0x7f988006fcf0 2024-04-22T05:50:42.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7f9875ffb640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9864066760 0x7f9864068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7f9875ffb640 1 -- 172.21.15.40:0/3973915296 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f987c085340 con 0x7f988006fcf0 2024-04-22T05:50:42.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff936ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff930032130 0x7ff930037920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff936ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff930032130 0x7ff930037920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:58928/0 (socket says 172.21.15.40:58928) 2024-04-22T05:50:42.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff936ffd640 1 -- 172.21.15.40:0/1022583281 learned_addr learned my addr 172.21.15.40:0/1022583281 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff9377fe640 1 --2- 172.21.15.40:0/1022583281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9300380f0 0x7ff9300b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7f9884951640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9864066760 0x7f9864068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe915bcf640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe90802fc10 0x7fe908161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe9163d0640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe908160bc0 0x7fe908162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe9153ce640 1 -- 172.21.15.40:0/726509246 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe90802fc10 msgr2=0x7fe908161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff9377fe640 1 -- 172.21.15.40:0/1022583281 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9300cf4d0 msgr2=0x7ff9300b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff9377fe640 1 --2- 172.21.15.40:0/1022583281 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9300cf4d0 0x7ff9300b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff9377fe640 1 -- 172.21.15.40:0/1022583281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff930032130 msgr2=0x7ff930037920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff9377fe640 1 --2- 172.21.15.40:0/1022583281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff930032130 0x7ff930037920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe9153ce640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe90802fc10 0x7fe908161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe9153ce640 1 -- 172.21.15.40:0/726509246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe908160bc0 msgr2=0x7fe908162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe9153ce640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe908160bc0 0x7fe908162490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe9153ce640 1 -- 172.21.15.40:0/726509246 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91004f100 con 0x7fe908034bf0 2024-04-22T05:50:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff9377fe640 1 -- 172.21.15.40:0/1022583281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93804f100 con 0x7ff9300380f0 2024-04-22T05:50:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff9377fe640 1 --2- 172.21.15.40:0/1022583281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9300380f0 0x7ff9300b9c50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff92800b3e0 tx=0x7ff92800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe9153ce640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe908034bf0 0x7fe9080c8210 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe910068240 tx=0x7fe9100946f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7f5b11d74640 1 -- 172.21.15.40:0/3284796339 shutdown_connections 2024-04-22T05:50:42.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7f5b11d74640 1 -- 172.21.15.40:0/3284796339 wait complete. 2024-04-22T05:50:42.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7f4aadd9b640 1 --2- 172.21.15.40:0/875442815 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4a90066690 0x7f4a90068b50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4007fd0 tx=0x7f4aa4007960 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7f9884951640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9864066760 0x7f9864068c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f98780ae000 tx=0x7f9878004ac0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe9067fc640 1 -- 172.21.15.40:0/726509246 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910126d40 con 0x7fe908034bf0 2024-04-22T05:50:42.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe9067fc640 1 -- 172.21.15.40:0/726509246 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91012f440 con 0x7fe908034bf0 2024-04-22T05:50:42.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe9067fc640 1 -- 172.21.15.40:0/726509246 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910126ea0 con 0x7fe908034bf0 2024-04-22T05:50:42.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe916bd1640 1 -- 172.21.15.40:0/726509246 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9080c8c40 con 0x7fe908034bf0 2024-04-22T05:50:42.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe916bd1640 1 -- 172.21.15.40:0/726509246 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe908162ba0 con 0x7fe908034bf0 2024-04-22T05:50:42.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7f5b11d74640 1 Processor -- start 2024-04-22T05:50:42.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7fe916bd1640 1 -- 172.21.15.40:0/726509246 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9081649c0 con 0x7fe908034bf0 2024-04-22T05:50:42.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7f5b11d74640 1 -- start start 2024-04-22T05:50:42.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff91f7fe640 1 -- 172.21.15.40:0/1022583281 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928011830 con 0x7ff9300380f0 2024-04-22T05:50:42.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff91f7fe640 1 -- 172.21.15.40:0/1022583281 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9280119b0 con 0x7ff9300380f0 2024-04-22T05:50:42.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff91f7fe640 1 -- 172.21.15.40:0/1022583281 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928011de0 con 0x7ff9300380f0 2024-04-22T05:50:42.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff937fff640 1 -- 172.21.15.40:0/1022583281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff930038aa0 con 0x7ff9300380f0 2024-04-22T05:50:42.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.570+0000 7ff937fff640 1 -- 172.21.15.40:0/1022583281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9300c9f20 con 0x7ff9300380f0 2024-04-22T05:50:42.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b11d74640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b0c06f3c0 0x7f5b0c12e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b11d74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b0c0713f0 0x7f5b0c135e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b11d74640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b0c0acb80 0x7f5b0c147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b11d74640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c1431d0 con 0x7f5b0c0713f0 2024-04-22T05:50:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b11d74640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c12ed20 con 0x7f5b0c0acb80 2024-04-22T05:50:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b11d74640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c136340 con 0x7f5b0c06f3c0 2024-04-22T05:50:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b0affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b0c0713f0 0x7f5b0c135e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7ff937fff640 1 -- 172.21.15.40:0/1022583281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9300387a0 con 0x7ff9300380f0 2024-04-22T05:50:42.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f9875ffb640 1 -- 172.21.15.40:0/3973915296 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f987c0529c0 con 0x7f988006fcf0 2024-04-22T05:50:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b0affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b0c0713f0 0x7f5b0c135e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59686/0 (socket says 172.21.15.40:59686) 2024-04-22T05:50:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b0affd640 1 -- 172.21.15.40:0/3835241168 learned_addr learned my addr 172.21.15.40:0/3835241168 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b0b7fe640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b0c06f3c0 0x7f5b0c12e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b0bfff640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b0c0acb80 0x7f5b0c147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b0affd640 1 -- 172.21.15.40:0/3835241168 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b0c06f3c0 msgr2=0x7f5b0c12e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b0affd640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b0c06f3c0 0x7f5b0c12e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b0affd640 1 -- 172.21.15.40:0/3835241168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b0c0acb80 msgr2=0x7f5b0c147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b0affd640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b0c0acb80 0x7f5b0c147560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b0affd640 1 -- 172.21.15.40:0/3835241168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b000094d0 con 0x7f5b0c0713f0 2024-04-22T05:50:42.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7f5b0affd640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b0c0713f0 0x7f5b0c135e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5b04007e90 tx=0x7f5b0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7fe9067fc640 1 -- 172.21.15.40:0/726509246 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe91012d070 con 0x7fe908034bf0 2024-04-22T05:50:42.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7ff3a1ffb640 1 -- 172.21.15.40:0/957244115 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff3a80a9070 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7fe9067fc640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe8f40666e0 0x7fe8f4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7fe9067fc640 1 -- 172.21.15.40:0/726509246 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe9101975a0 con 0x7fe908034bf0 2024-04-22T05:50:42.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7fe915bcf640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe8f40666e0 0x7fe8f4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7fe8f67fc640 1 -- 172.21.15.40:0/744734911 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc009690 con 0x7fe900071860 2024-04-22T05:50:42.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7fe907cca640 1 -- 172.21.15.40:0/744734911 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9000ab790 con 0x7fe900071860 2024-04-22T05:50:42.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7fe907cca640 1 -- 172.21.15.40:0/744734911 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9001d8f90 con 0x7fe900071860 2024-04-22T05:50:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7ff3a1ffb640 1 --2- 172.21.15.40:0/957244115 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff3900666e0 0x7ff390068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7ff91f7fe640 1 -- 172.21.15.40:0/1022583281 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff92801a400 con 0x7ff9300380f0 2024-04-22T05:50:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7ff3a1ffb640 1 -- 172.21.15.40:0/957244115 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff3a811e830 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7fe915bcf640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe8f40666e0 0x7fe8f4068ba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe900001800 tx=0x7fe900009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7ff91f7fe640 1 --2- 172.21.15.40:0/1022583281 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff924066760 0x7ff924068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7ff91f7fe640 1 -- 172.21.15.40:0/1022583281 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff928084550 con 0x7ff9300380f0 2024-04-22T05:50:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7ff936ffd640 1 --2- 172.21.15.40:0/1022583281 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff924066760 0x7ff924068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7fe907cca640 1 -- 172.21.15.40:0/744734911 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9001d8580 con 0x7fe900071860 2024-04-22T05:50:42.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7fe8f67fc640 1 -- 172.21.15.40:0/744734911 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc00ca00 con 0x7fe900071860 2024-04-22T05:50:42.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7fe8f67fc640 1 -- 172.21.15.40:0/744734911 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc00cd70 con 0x7fe900071860 2024-04-22T05:50:42.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.574+0000 7ff936ffd640 1 --2- 172.21.15.40:0/1022583281 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff924066760 0x7ff924068c20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff938072e20 tx=0x7ff938120000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.578+0000 7ff3b10ad640 1 --2- 172.21.15.40:0/957244115 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff3900666e0 0x7ff390068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.578+0000 7ff3b10ad640 1 --2- 172.21.15.40:0/957244115 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff3900666e0 0x7ff390068ba0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009c20 tx=0x7ff39c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.578+0000 7f5af3fff640 1 -- 172.21.15.40:0/3835241168 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04014420 con 0x7f5b0c0713f0 2024-04-22T05:50:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.578+0000 7f5b11d74640 1 -- 172.21.15.40:0/3835241168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c1482b0 con 0x7f5b0c0713f0 2024-04-22T05:50:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.578+0000 7f5b11d74640 1 -- 172.21.15.40:0/3835241168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c130170 con 0x7f5b0c0713f0 2024-04-22T05:50:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.578+0000 7ff91f7fe640 1 -- 172.21.15.40:0/1022583281 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7ff928051b20 con 0x7ff9300380f0 2024-04-22T05:50:42.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.578+0000 7f5b11d74640 1 -- 172.21.15.40:0/3835241168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0c0702a0 con 0x7f5b0c0713f0 2024-04-22T05:50:42.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.582+0000 7fe4cb7fe640 1 -- 172.21.15.40:0/4208858245 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8004230 con 0x7fe4dc035c40 2024-04-22T05:50:42.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.582+0000 7fe4cb7fe640 1 -- 172.21.15.40:0/4208858245 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d800c040 con 0x7fe4dc035c40 2024-04-22T05:50:42.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.582+0000 7fe4e3fff640 1 -- 172.21.15.40:0/4208858245 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4dc161140 con 0x7fe4dc035c40 2024-04-22T05:50:42.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.582+0000 7fe4cb7fe640 1 -- 172.21.15.40:0/4208858245 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d800d920 con 0x7fe4dc035c40 2024-04-22T05:50:42.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.582+0000 7fe4e3fff640 1 -- 172.21.15.40:0/4208858245 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc161640 con 0x7fe4dc035c40 2024-04-22T05:50:42.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.582+0000 7f4a9effd640 1 -- 172.21.15.40:0/875442815 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f4aa0052290 con 0x7f4ab006e9d0 2024-04-22T05:50:42.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.582+0000 7fe4e3fff640 1 -- 172.21.15.40:0/4208858245 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c4005050 con 0x7fe4dc035c40 2024-04-22T05:50:42.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.582+0000 7ff3a1ffb640 1 -- 172.21.15.40:0/957244115 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7ff3a80ebf20 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.582+0000 7fe8f67fc640 1 -- 172.21.15.40:0/744734911 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe8fc013470 con 0x7fe900071860 2024-04-22T05:50:42.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.582+0000 7fe8f67fc640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe8e4066690 0x7fe8e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.586+0000 7fe905a66640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe8e4066690 0x7fe8e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.586+0000 7fe8f67fc640 1 -- 172.21.15.40:0/744734911 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe8fc055cc0 con 0x7fe900071860 2024-04-22T05:50:42.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.586+0000 7fe4cb7fe640 1 -- 172.21.15.40:0/4208858245 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe4d800da80 con 0x7fe4dc035c40 2024-04-22T05:50:42.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.586+0000 7fe4cb7fe640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe4d0066710 0x7fe4d0068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.586+0000 7fe4cb7fe640 1 -- 172.21.15.40:0/4208858245 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe4d8056180 con 0x7fe4dc035c40 2024-04-22T05:50:42.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.586+0000 7fe4e2ffd640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe4d0066710 0x7fe4d0068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.586+0000 7fe9067fc640 1 -- 172.21.15.40:0/726509246 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7fe9081649c0 con 0x7fe908034bf0 2024-04-22T05:50:42.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.586+0000 7fe905a66640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe8e4066690 0x7fe8e4068b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00ae000 tx=0x7fe8f00046d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.586+0000 7fe4e2ffd640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe4d0066710 0x7fe4d0068bd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40634c0 tx=0x7fe4e4066c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.586+0000 7f5af3fff640 1 -- 172.21.15.40:0/3835241168 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b04009980 con 0x7f5b0c0713f0 2024-04-22T05:50:42.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.586+0000 7f5af3fff640 1 -- 172.21.15.40:0/3835241168 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b040147a0 con 0x7f5b0c0713f0 2024-04-22T05:50:42.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.590+0000 7f5af3fff640 1 -- 172.21.15.40:0/3835241168 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5b04014900 con 0x7f5b0c0713f0 2024-04-22T05:50:42.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.590+0000 7f5af3fff640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f5af8066690 0x7f5af8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:42.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.590+0000 7f5af3fff640 1 -- 172.21.15.40:0/3835241168 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5b04085610 con 0x7f5b0c0713f0 2024-04-22T05:50:42.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.590+0000 7fe8f67fc640 1 -- 172.21.15.40:0/744734911 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7fe8fc052000 con 0x7fe900071860 2024-04-22T05:50:42.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.598+0000 7f5af3fff640 1 -- 172.21.15.40:0/3835241168 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f5b04052c00 con 0x7f5b0c0713f0 2024-04-22T05:50:42.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.598+0000 7fe4cb7fe640 1 -- 172.21.15.40:0/4208858245 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7fe4d8051dc0 con 0x7fe4dc035c40 2024-04-22T05:50:42.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.602+0000 7f5b0b7fe640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f5af8066690 0x7f5af8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:42.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.602+0000 7f5b0b7fe640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f5af8066690 0x7f5af8068b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5b000ae000 tx=0x7f5b00008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:42.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.890+0000 7f98873b6640 1 -- 172.21.15.40:0/3973915296 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f988013ad30 con 0x7f988006fcf0 2024-04-22T05:50:42.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.890+0000 7f9875ffb640 1 -- 172.21.15.40:0/3973915296 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f987c055fe0 con 0x7f988006fcf0 2024-04-22T05:50:42.892 INFO:teuthology.orchestra.run.smithi040.stdout:60129542146 2024-04-22T05:50:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.894+0000 7f98637fe640 1 -- 172.21.15.40:0/3973915296 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9864066760 msgr2=0x7f9864068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.894+0000 7f98637fe640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9864066760 0x7f9864068c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f98780ae000 tx=0x7f9878004ac0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.894+0000 7f98637fe640 1 -- 172.21.15.40:0/3973915296 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f988006fcf0 msgr2=0x7f9880131a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.894+0000 7f98637fe640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f988006fcf0 0x7f9880131a50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f987c00cea0 tx=0x7f987c00cf90 comp rx=0 tx=0).stop 2024-04-22T05:50:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.894+0000 7f98637fe640 1 -- 172.21.15.40:0/3973915296 shutdown_connections 2024-04-22T05:50:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.894+0000 7f98637fe640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9864066760 0x7f9864068c20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.894+0000 7f98637fe640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98800716b0 0x7f98801ddfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.894+0000 7f98637fe640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9880140800 0x7f9880130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.894+0000 7f98637fe640 1 --2- 172.21.15.40:0/3973915296 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f988006fcf0 0x7f9880131a50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.894+0000 7f98637fe640 1 -- 172.21.15.40:0/3973915296 >> 172.21.15.40:0/3973915296 conn(0x7f98800b27a0 msgr2=0x7f98800b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.894+0000 7f98637fe640 1 -- 172.21.15.40:0/3973915296 shutdown_connections 2024-04-22T05:50:42.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.894+0000 7f98637fe640 1 -- 172.21.15.40:0/3973915296 wait complete. 2024-04-22T05:50:42.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.906+0000 7fe916bd1640 1 -- 172.21.15.40:0/726509246 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe9081649c0 con 0x7fe908034bf0 2024-04-22T05:50:42.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.910+0000 7fe9067fc640 1 -- 172.21.15.40:0/726509246 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe9081649c0 con 0x7fe908034bf0 2024-04-22T05:50:42.910 INFO:teuthology.orchestra.run.smithi040.stdout:60129542146 2024-04-22T05:50:42.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.910+0000 7ff937fff640 1 -- 172.21.15.40:0/1022583281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff9300052a0 con 0x7ff9300380f0 2024-04-22T05:50:42.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.910+0000 7ff91f7fe640 1 -- 172.21.15.40:0/1022583281 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff928055140 con 0x7ff9300380f0 2024-04-22T05:50:42.913 INFO:teuthology.orchestra.run.smithi040.stdout:60129542147 2024-04-22T05:50:42.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.910+0000 7fe8f3fff640 1 -- 172.21.15.40:0/726509246 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe8f40666e0 msgr2=0x7fe8f4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.910+0000 7fe8f3fff640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe8f40666e0 0x7fe8f4068ba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe900001800 tx=0x7fe900009040 comp rx=0 tx=0).stop 2024-04-22T05:50:42.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7fe8f3fff640 1 -- 172.21.15.40:0/726509246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe908034bf0 msgr2=0x7fe9080c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7fe8f3fff640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe908034bf0 0x7fe9080c8210 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe910068240 tx=0x7fe9100946f0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/875442815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4a94005300 con 0x7f4ab006e9d0 2024-04-22T05:50:42.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7fe8f3fff640 1 -- 172.21.15.40:0/726509246 shutdown_connections 2024-04-22T05:50:42.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7fe8f3fff640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe8f40666e0 0x7fe8f4068ba0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7fe8f3fff640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe908160bc0 0x7fe908162490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7fe8f3fff640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe908034bf0 0x7fe9080c8210 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7fe8f3fff640 1 --2- 172.21.15.40:0/726509246 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe90802fc10 0x7fe908161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7fe8f3fff640 1 -- 172.21.15.40:0/726509246 >> 172.21.15.40:0/726509246 conn(0x7fe908006090 msgr2=0x7fe9080340d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7fe8f3fff640 1 -- 172.21.15.40:0/726509246 shutdown_connections 2024-04-22T05:50:42.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7fe8f3fff640 1 -- 172.21.15.40:0/726509246 wait complete. 2024-04-22T05:50:42.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7f4a9effd640 1 -- 172.21.15.40:0/875442815 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4aa00558b0 con 0x7f4ab006e9d0 2024-04-22T05:50:42.917 INFO:teuthology.orchestra.run.smithi040.stdout:55834574849 2024-04-22T05:50:42.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7ff3977fe640 1 -- 172.21.15.40:0/957244115 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff38c0055d0 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7ff937fff640 1 -- 172.21.15.40:0/1022583281 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff924066760 msgr2=0x7ff924068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7ff937fff640 1 --2- 172.21.15.40:0/1022583281 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff924066760 0x7ff924068c20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff938072e20 tx=0x7ff938120000 comp rx=0 tx=0).stop 2024-04-22T05:50:42.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7ff937fff640 1 -- 172.21.15.40:0/1022583281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9300380f0 msgr2=0x7ff9300b9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7ff937fff640 1 --2- 172.21.15.40:0/1022583281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9300380f0 0x7ff9300b9c50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff92800b3e0 tx=0x7ff92800b8a0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.918 INFO:teuthology.orchestra.run.smithi040.stdout:60129542147 2024-04-22T05:50:42.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.914+0000 7ff3a1ffb640 1 -- 172.21.15.40:0/957244115 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff3a80ef540 con 0x7ff3ac0af6b0 2024-04-22T05:50:42.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7ff937fff640 1 -- 172.21.15.40:0/1022583281 shutdown_connections 2024-04-22T05:50:42.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7ff937fff640 1 --2- 172.21.15.40:0/1022583281 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff924066760 0x7ff924068c20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7ff937fff640 1 --2- 172.21.15.40:0/1022583281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9300380f0 0x7ff9300b9c50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7ff937fff640 1 --2- 172.21.15.40:0/1022583281 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9300cf4d0 0x7ff9300b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7ff937fff640 1 --2- 172.21.15.40:0/1022583281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff930032130 0x7ff930037920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7ff937fff640 1 -- 172.21.15.40:0/1022583281 >> 172.21.15.40:0/1022583281 conn(0x7ff930005ec0 msgr2=0x7ff93002f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7ff937fff640 1 -- 172.21.15.40:0/1022583281 shutdown_connections 2024-04-22T05:50:42.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7ff937fff640 1 -- 172.21.15.40:0/1022583281 wait complete. 2024-04-22T05:50:42.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.0 2024-04-22T05:50:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/875442815 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4a90066690 msgr2=0x7f4a90068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7f4ab4bb6640 1 --2- 172.21.15.40:0/875442815 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4a90066690 0x7f4a90068b50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4007fd0 tx=0x7f4aa4007960 comp rx=0 tx=0).stop 2024-04-22T05:50:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/875442815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ab006e9d0 msgr2=0x7f4ab01dd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7f4ab4bb6640 1 --2- 172.21.15.40:0/875442815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ab006e9d0 0x7f4ab01dd270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009db0 tx=0x7f4aa000c490 comp rx=0 tx=0).stop 2024-04-22T05:50:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/875442815 shutdown_connections 2024-04-22T05:50:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7f4ab4bb6640 1 --2- 172.21.15.40:0/875442815 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4a90066690 0x7f4a90068b50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7f4ab4bb6640 1 --2- 172.21.15.40:0/875442815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ab0070ed0 0x7f4ab01d9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7f4ab4bb6640 1 --2- 172.21.15.40:0/875442815 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4ab006f350 0x7f4ab01dd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7f4ab4bb6640 1 --2- 172.21.15.40:0/875442815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ab006e9d0 0x7f4ab01dd270 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/875442815 >> 172.21.15.40:0/875442815 conn(0x7f4ab00b27a0 msgr2=0x7f4ab00b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/875442815 shutdown_connections 2024-04-22T05:50:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7f4ab4bb6640 1 -- 172.21.15.40:0/875442815 wait complete. 2024-04-22T05:50:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7ff3977fe640 1 -- 172.21.15.40:0/957244115 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff3900666e0 msgr2=0x7ff390068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7ff3977fe640 1 --2- 172.21.15.40:0/957244115 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff3900666e0 0x7ff390068ba0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009c20 tx=0x7ff39c009190 comp rx=0 tx=0).stop 2024-04-22T05:50:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7ff3977fe640 1 -- 172.21.15.40:0/957244115 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff3ac0af6b0 msgr2=0x7ff3ac1416f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7ff3977fe640 1 --2- 172.21.15.40:0/957244115 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff3ac0af6b0 0x7ff3ac1416f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80ac000 tx=0x7ff3a801ac80 comp rx=0 tx=0).stop 2024-04-22T05:50:42.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7fe907cca640 1 -- 172.21.15.40:0/744734911 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe9001363e0 con 0x7fe900071860 2024-04-22T05:50:42.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.918+0000 7fe8f67fc640 1 -- 172.21.15.40:0/744734911 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe8fc021020 con 0x7fe900071860 2024-04-22T05:50:42.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7ff3977fe640 1 -- 172.21.15.40:0/957244115 shutdown_connections 2024-04-22T05:50:42.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7ff3977fe640 1 --2- 172.21.15.40:0/957244115 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff3900666e0 0x7ff390068ba0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7ff3977fe640 1 --2- 172.21.15.40:0/957244115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac0b05a0 0x7ff3ac141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7ff3977fe640 1 --2- 172.21.15.40:0/957244115 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff3ac0af6b0 0x7ff3ac1416f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7ff3977fe640 1 --2- 172.21.15.40:0/957244115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac06c0f0 0x7ff3ac147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7ff3977fe640 1 -- 172.21.15.40:0/957244115 >> 172.21.15.40:0/957244115 conn(0x7ff3ac0b49a0 msgr2=0x7ff3ac06fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.925 INFO:teuthology.orchestra.run.smithi040.stdout:60129542145 2024-04-22T05:50:42.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7fe907cca640 1 -- 172.21.15.40:0/744734911 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe8e4066690 msgr2=0x7fe8e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7fe907cca640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe8e4066690 0x7fe8e4068b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00ae000 tx=0x7fe8f00046d0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7fe907cca640 1 -- 172.21.15.40:0/744734911 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe900071860 msgr2=0x7fe9001d8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7fe907cca640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe900071860 0x7fe9001d8880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00e930 tx=0x7fe8fc00edf0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7fe907cca640 1 -- 172.21.15.40:0/744734911 shutdown_connections 2024-04-22T05:50:42.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7fe907cca640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe8e4066690 0x7fe8e4068b50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7fe907cca640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9000afc10 0x7fe9001469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7fe907cca640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe900071860 0x7fe9001d8880 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7fe907cca640 1 --2- 172.21.15.40:0/744734911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe90006eed0 0x7fe9000ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7fe907cca640 1 -- 172.21.15.40:0/744734911 >> 172.21.15.40:0/744734911 conn(0x7fe9000b2da0 msgr2=0x7fe9000b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7fe907cca640 1 -- 172.21.15.40:0/744734911 shutdown_connections 2024-04-22T05:50:42.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.922+0000 7f5b11d74640 1 -- 172.21.15.40:0/3835241168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5b0c067570 con 0x7f5b0c0713f0 2024-04-22T05:50:42.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7f5af3fff640 1 -- 172.21.15.40:0/3835241168 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5b04056220 con 0x7f5b0c0713f0 2024-04-22T05:50:42.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7ff3977fe640 1 -- 172.21.15.40:0/957244115 shutdown_connections 2024-04-22T05:50:42.927 INFO:teuthology.orchestra.run.smithi040.stdout:55834574851 2024-04-22T05:50:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7ff3977fe640 1 -- 172.21.15.40:0/957244115 wait complete. 2024-04-22T05:50:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7fe907cca640 1 -- 172.21.15.40:0/744734911 wait complete. 2024-04-22T05:50:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7fe4e3fff640 1 -- 172.21.15.40:0/4208858245 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe4c40055d0 con 0x7fe4dc035c40 2024-04-22T05:50:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7fe4cb7fe640 1 -- 172.21.15.40:0/4208858245 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe4d8020020 con 0x7fe4dc035c40 2024-04-22T05:50:42.930 INFO:teuthology.orchestra.run.smithi040.stdout:55834574850 2024-04-22T05:50:42.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7f5b11d74640 1 -- 172.21.15.40:0/3835241168 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f5af8066690 msgr2=0x7f5af8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7f5b11d74640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f5af8066690 0x7f5af8068b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5b000ae000 tx=0x7f5b00008040 comp rx=0 tx=0).stop 2024-04-22T05:50:42.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7f5b11d74640 1 -- 172.21.15.40:0/3835241168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b0c0713f0 msgr2=0x7f5b0c135e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7f5b11d74640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b0c0713f0 0x7f5b0c135e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5b04007e90 tx=0x7f5b0400e4b0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7f5b11d74640 1 -- 172.21.15.40:0/3835241168 shutdown_connections 2024-04-22T05:50:42.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7f5b11d74640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f5af8066690 0x7f5af8068b50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7f5b11d74640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b0c0acb80 0x7f5b0c147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7f5b11d74640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b0c0713f0 0x7f5b0c135e00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7f5b11d74640 1 --2- 172.21.15.40:0/3835241168 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f5b0c06f3c0 0x7f5b0c12e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7f5b11d74640 1 -- 172.21.15.40:0/3835241168 >> 172.21.15.40:0/3835241168 conn(0x7f5b0c0b27a0 msgr2=0x7f5b0c0b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7f5b11d74640 1 -- 172.21.15.40:0/3835241168 shutdown_connections 2024-04-22T05:50:42.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.926+0000 7f5b11d74640 1 -- 172.21.15.40:0/3835241168 wait complete. 2024-04-22T05:50:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.938+0000 7fe4e3fff640 1 -- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe4d0066710 msgr2=0x7fe4d0068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.938+0000 7fe4e3fff640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe4d0066710 0x7fe4d0068bd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40634c0 tx=0x7fe4e4066c90 comp rx=0 tx=0).stop 2024-04-22T05:50:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.938+0000 7fe4e3fff640 1 -- 172.21.15.40:0/4208858245 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4dc035c40 msgr2=0x7fe4dc0c46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.938+0000 7fe4e3fff640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4dc035c40 0x7fe4dc0c46a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800a4c0 tx=0x7fe4d800fd80 comp rx=0 tx=0).stop 2024-04-22T05:50:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.938+0000 7fe4e3fff640 1 -- 172.21.15.40:0/4208858245 shutdown_connections 2024-04-22T05:50:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.938+0000 7fe4e3fff640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe4d0066710 0x7fe4d0068bd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.938+0000 7fe4e3fff640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4dc160290 0x7fe4dc160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.938+0000 7fe4e3fff640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe4dc035c40 0x7fe4dc0c46a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.938+0000 7fe4e3fff640 1 --2- 172.21.15.40:0/4208858245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4cc003980 0x7fe4dc0c3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.938+0000 7fe4e3fff640 1 -- 172.21.15.40:0/4208858245 >> 172.21.15.40:0/4208858245 conn(0x7fe4dc03a850 msgr2=0x7fe4dc0c0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:42.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.938+0000 7fe4e3fff640 1 -- 172.21.15.40:0/4208858245 shutdown_connections 2024-04-22T05:50:42.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:42.938+0000 7fe4e3fff640 1 -- 172.21.15.40:0/4208858245 wait complete. 2024-04-22T05:50:42.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.5 2024-04-22T05:50:42.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.4 2024-04-22T05:50:42.956 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.1 2024-04-22T05:50:42.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.6 2024-04-22T05:50:42.961 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542146 got 60129542145 for osd.2 2024-04-22T05:50:42.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.3 2024-04-22T05:50:42.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2024-04-22T05:50:43.922 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-22T05:50:43.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-22T05:50:43.953 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-22T05:50:43.956 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-22T05:50:43.960 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-22T05:50:43.961 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-22T05:50:43.963 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-22T05:50:43.967 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-22T05:50:44.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.086+0000 7f06b7828640 1 Processor -- start 2024-04-22T05:50:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.086+0000 7f06b7828640 1 -- start start 2024-04-22T05:50:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.090+0000 7f06b7828640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f06b0071d60 0x7f06b0148660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.090+0000 7f06b7828640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b0071a90 con 0x7f06b00716b0 2024-04-22T05:50:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.090+0000 7f06b7828640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b0070cc0 con 0x7f06b006fcf0 2024-04-22T05:50:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.090+0000 7f06b7828640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b006ed70 con 0x7f06b0071d60 2024-04-22T05:50:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.090+0000 7f06b55c4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f06b00716b0 0x7f06b01402c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56054/0 (socket says 172.21.15.40:56054) 2024-04-22T05:50:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.090+0000 7f06b55c4640 1 -- 172.21.15.40:0/2613915212 learned_addr learned my addr 172.21.15.40:0/2613915212 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:50:44.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.090+0000 7f06b5dc5640 1 --2- 172.21.15.40:0/2613915212 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f06b0071d60 0x7f06b0148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.090+0000 7f06a7fff640 1 -- 172.21.15.40:0/2613915212 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452711416 0 0) 0x7f06b0071a90 con 0x7f06b00716b0 2024-04-22T05:50:44.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.090+0000 7f06a7fff640 1 -- 172.21.15.40:0/2613915212 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f069c003530 con 0x7f06b00716b0 2024-04-22T05:50:44.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.090+0000 7f06b5dc5640 1 -- 172.21.15.40:0/2613915212 >> v1:172.21.15.62:6789/0 conn(0x7f06b006fcf0 legacy=0x7f06b00ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.090+0000 7f06b5dc5640 1 -- 172.21.15.40:0/2613915212 >> v1:172.21.15.40:6789/0 conn(0x7f06b00716b0 legacy=0x7f06b01402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.090+0000 7f06b5dc5640 1 -- 172.21.15.40:0/2613915212 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b006fa30 con 0x7f06b0071d60 2024-04-22T05:50:44.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.094+0000 7f06b5dc5640 1 --2- 172.21.15.40:0/2613915212 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f06b0071d60 0x7f06b0148660 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f06a000d8d0 tx=0x7f06a001e210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a286fcdc0e71808 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.094+0000 7f06a7fff640 1 -- 172.21.15.40:0/2613915212 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a00a6a70 con 0x7f06b0071d60 2024-04-22T05:50:44.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.094+0000 7f06a7fff640 1 -- 172.21.15.40:0/2613915212 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a00a6bd0 con 0x7f06b0071d60 2024-04-22T05:50:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.094+0000 7f06b7828640 1 -- 172.21.15.40:0/2613915212 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f06b0071d60 msgr2=0x7f06b0148660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.094+0000 7f06b7828640 1 --2- 172.21.15.40:0/2613915212 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f06b0071d60 0x7f06b0148660 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f06a000d8d0 tx=0x7f06a001e210 comp rx=0 tx=0).stop 2024-04-22T05:50:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.094+0000 7f06b7828640 1 -- 172.21.15.40:0/2613915212 shutdown_connections 2024-04-22T05:50:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.094+0000 7f06b7828640 1 --2- 172.21.15.40:0/2613915212 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f06b0071d60 0x7f06b0148660 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.094+0000 7f06b7828640 1 -- 172.21.15.40:0/2613915212 >> 172.21.15.40:0/2613915212 conn(0x7f06b00b27a0 msgr2=0x7f06b00b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.094+0000 7f06b7828640 1 -- 172.21.15.40:0/2613915212 shutdown_connections 2024-04-22T05:50:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.094+0000 7f06b7828640 1 -- 172.21.15.40:0/2613915212 wait complete. 2024-04-22T05:50:44.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b7828640 1 Processor -- start 2024-04-22T05:50:44.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b7828640 1 -- start start 2024-04-22T05:50:44.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b7828640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f06b006fcf0 0x7f06b00abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b7828640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00716b0 0x7f06b0136630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b7828640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b0071d60 0x7f06b0131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b7828640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b0136b70 con 0x7f06b0071d60 2024-04-22T05:50:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b7828640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b0136de0 con 0x7f06b00716b0 2024-04-22T05:50:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b7828640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b0135b70 con 0x7f06b006fcf0 2024-04-22T05:50:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b5dc5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b0071d60 0x7f06b0131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b5dc5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b0071d60 0x7f06b0131840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59700/0 (socket says 172.21.15.40:59700) 2024-04-22T05:50:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b5dc5640 1 -- 172.21.15.40:0/1378899348 learned_addr learned my addr 172.21.15.40:0/1378899348 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b4dc3640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00716b0 0x7f06b0136630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b55c4640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f06b006fcf0 0x7f06b00abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b5dc5640 1 -- 172.21.15.40:0/1378899348 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f06b006fcf0 msgr2=0x7f06b00abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b5dc5640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f06b006fcf0 0x7f06b00abac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b5dc5640 1 -- 172.21.15.40:0/1378899348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00716b0 msgr2=0x7f06b0136630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b5dc5640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00716b0 0x7f06b0136630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b5dc5640 1 -- 172.21.15.40:0/1378899348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a000d510 con 0x7f06b0071d60 2024-04-22T05:50:44.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.098+0000 7f06b5dc5640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b0071d60 0x7f06b0131840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f06a001c080 tx=0x7f06a001b300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.106+0000 7f06a5ffb640 1 -- 172.21.15.40:0/1378899348 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a00a6870 con 0x7f06b0071d60 2024-04-22T05:50:44.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.106+0000 7f06b7828640 1 -- 172.21.15.40:0/1378899348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b0135d90 con 0x7f06b0071d60 2024-04-22T05:50:44.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.106+0000 7f06b7828640 1 -- 172.21.15.40:0/1378899348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b0131f50 con 0x7f06b0071d60 2024-04-22T05:50:44.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.106+0000 7f06b7828640 1 -- 172.21.15.40:0/1378899348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b0067570 con 0x7f06b0071d60 2024-04-22T05:50:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.106+0000 7f06a5ffb640 1 -- 172.21.15.40:0/1378899348 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a00a69d0 con 0x7f06b0071d60 2024-04-22T05:50:44.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.118+0000 7f06a5ffb640 1 -- 172.21.15.40:0/1378899348 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a00b9660 con 0x7f06b0071d60 2024-04-22T05:50:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.118+0000 7f06a5ffb640 1 -- 172.21.15.40:0/1378899348 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f06a00b5040 con 0x7f06b0071d60 2024-04-22T05:50:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.118+0000 7f06a5ffb640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f06940667e0 0x7f0694068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.118+0000 7f06a5ffb640 1 -- 172.21.15.40:0/1378899348 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f06a011dcb0 con 0x7f06b0071d60 2024-04-22T05:50:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.118+0000 7f06a5ffb640 1 -- 172.21.15.40:0/1378899348 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f06a01212a0 con 0x7f06b0071d60 2024-04-22T05:50:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.118+0000 7f06b55c4640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f06940667e0 0x7f0694068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.130+0000 7f06b55c4640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f06940667e0 0x7f0694068ca0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f06a8004e10 tx=0x7f06a8007500 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.150+0000 7f7d88fe1640 1 Processor -- start 2024-04-22T05:50:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.150+0000 7f7d88fe1640 1 -- start start 2024-04-22T05:50:44.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a56f91640 1 Processor -- start 2024-04-22T05:50:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b1808e640 1 Processor -- start 2024-04-22T05:50:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a56f91640 1 -- start start 2024-04-22T05:50:44.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d88fe1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d840b4210 0x7f7d840aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d88fe1640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7d8413e1f0 0x7f7d8413e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d88fe1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d840b45f0 con 0x7f7d840ad1b0 2024-04-22T05:50:44.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d88fe1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d840ac860 con 0x7f7d840b4210 2024-04-22T05:50:44.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d88fe1640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d840acab0 con 0x7f7d8413e1f0 2024-04-22T05:50:44.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d837fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d840b4210 0x7f7d840aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d837fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d840b4210 0x7f7d840aeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59716/0 (socket says 172.21.15.40:59716) 2024-04-22T05:50:44.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d837fe640 1 -- 172.21.15.40:0/4205624072 learned_addr learned my addr 172.21.15.40:0/4205624072 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d82ffd640 1 --2- 172.21.15.40:0/4205624072 >> v2:172.21.15.62:3300/0 conn(0x7f7d8413e1f0 0x7f7d8413e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d837fe640 1 -- 172.21.15.40:0/4205624072 >> v2:172.21.15.62:3300/0 conn(0x7f7d8413e1f0 msgr2=0x7f7d8413e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:44.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d837fe640 1 --2- 172.21.15.40:0/4205624072 >> v2:172.21.15.62:3300/0 conn(0x7f7d8413e1f0 0x7f7d8413e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b1808e640 1 -- start start 2024-04-22T05:50:44.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b1808e640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6b100b0c90 0x7f6b1006ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b1808e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b100afe00 0x7f6b10138890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b1808e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1013f290 con 0x7f6b100afa20 2024-04-22T05:50:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b1808e640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b100a7bb0 con 0x7f6b100b0c90 2024-04-22T05:50:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b1808e640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100a7e90 con 0x7f6b100afe00 2024-04-22T05:50:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b15629640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6b100b0c90 0x7f6b1006ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b1662b640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b100afe00 0x7f6b10138890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b15629640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6b100b0c90 0x7f6b1006ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:58964/0 (socket says 172.21.15.40:58964) 2024-04-22T05:50:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b15629640 1 -- 172.21.15.40:0/3200671639 learned_addr learned my addr 172.21.15.40:0/3200671639 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a56f91640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a48037500 0x7f6a480c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a56f91640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a4802c4f0 0x7f6a480c1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a55f8f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a48037500 0x7f6a480c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a55f8f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a48037500 0x7f6a480c1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59720/0 (socket says 172.21.15.40:59720) 2024-04-22T05:50:44.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a56f91640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a480378e0 con 0x7f6a480c7dc0 2024-04-22T05:50:44.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a56f91640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4802d830 con 0x7f6a48037500 2024-04-22T05:50:44.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a56f91640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4802c0a0 con 0x7f6a4802c4f0 2024-04-22T05:50:44.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a5578e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a4802c4f0 0x7f6a480c1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a5578e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a4802c4f0 0x7f6a480c1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35256/0 (socket says 172.21.15.40:35256) 2024-04-22T05:50:44.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a5578e640 1 -- 172.21.15.40:0/3715684112 learned_addr learned my addr 172.21.15.40:0/3715684112 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d837fe640 1 -- 172.21.15.40:0/4205624072 >> v1:172.21.15.40:6789/0 conn(0x7f7d840ad1b0 legacy=0x7f7d840ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d827fc640 1 -- 172.21.15.40:0/4205624072 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228935964 0 0) 0x7f7d840b45f0 con 0x7f7d840ad1b0 2024-04-22T05:50:44.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f66de70c640 1 Processor -- start 2024-04-22T05:50:44.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f66de70c640 1 -- start start 2024-04-22T05:50:44.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f7d837fe640 1 -- 172.21.15.40:0/4205624072 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d840acd90 con 0x7f7d840b4210 2024-04-22T05:50:44.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a5578e640 1 -- 172.21.15.40:0/3715684112 >> v2:172.21.15.40:3300/0 conn(0x7f6a48037500 msgr2=0x7f6a480c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:44.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a5578e640 1 --2- 172.21.15.40:0/3715684112 >> v2:172.21.15.40:3300/0 conn(0x7f6a48037500 0x7f6a480c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a5578e640 1 -- 172.21.15.40:0/3715684112 >> v1:172.21.15.40:6789/0 conn(0x7f6a480c7dc0 legacy=0x7f6a480c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6a5578e640 1 -- 172.21.15.40:0/3715684112 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4802b400 con 0x7f6a4802c4f0 2024-04-22T05:50:44.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66de70c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f66d806f070 0x7f66d813e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66de70c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f66d80701a0 0x7f66d80ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66de70c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66d806f450 0x7f66d813af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66de70c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d813ec00 con 0x7f66d806f070 2024-04-22T05:50:44.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66de70c640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d8071270 con 0x7f66d80701a0 2024-04-22T05:50:44.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66de70c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d813e1a0 con 0x7f66d806f450 2024-04-22T05:50:44.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66d77fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f66d80701a0 0x7f66d80ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66d77fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f66d80701a0 0x7f66d80ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:58970/0 (socket says 172.21.15.40:58970) 2024-04-22T05:50:44.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66d77fe640 1 -- 172.21.15.40:0/560357895 learned_addr learned my addr 172.21.15.40:0/560357895 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66d77fe640 1 -- 172.21.15.40:0/560357895 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66d806f450 msgr2=0x7f66d813af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:44.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66d77fe640 1 --2- 172.21.15.40:0/560357895 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66d806f450 0x7f66d813af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66d77fe640 1 -- 172.21.15.40:0/560357895 >> v2:172.21.15.40:3300/0 conn(0x7f66d806f070 msgr2=0x7f66d813e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:44.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66d77fe640 1 --2- 172.21.15.40:0/560357895 >> v2:172.21.15.40:3300/0 conn(0x7f66d806f070 0x7f66d813e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f66d77fe640 1 -- 172.21.15.40:0/560357895 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d8071970 con 0x7f66d80701a0 2024-04-22T05:50:44.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b15629640 1 -- 172.21.15.40:0/3200671639 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b100afe00 msgr2=0x7f6b10138890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:44.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b15629640 1 --2- 172.21.15.40:0/3200671639 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b100afe00 0x7f6b10138890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b15629640 1 -- 172.21.15.40:0/3200671639 >> v1:172.21.15.40:6789/0 conn(0x7f6b100afa20 legacy=0x7f6b1013f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.154+0000 7f6b15629640 1 -- 172.21.15.40:0/3200671639 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b10070440 con 0x7f6b100b0c90 2024-04-22T05:50:44.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f9574aa4640 1 Processor -- start 2024-04-22T05:50:44.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6b15629640 1 --2- 172.21.15.40:0/3200671639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b100b0c90 0x7f6b1006ff00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009870 tx=0x7f6b0c01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c779ae4fed62b03a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f9574aa4640 1 -- start start 2024-04-22T05:50:44.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec2a0f640 1 Processor -- start 2024-04-22T05:50:44.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec2a0f640 1 -- start start 2024-04-22T05:50:44.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6b14e28640 1 -- 172.21.15.40:0/3200671639 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0aa650 con 0x7f6b100b0c90 2024-04-22T05:50:44.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f7d837fe640 1 --2- 172.21.15.40:0/4205624072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d840b4210 0x7f7d840aeff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7d84141f60 tx=0x7f7d7401c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c40c587d00366e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6b14e28640 1 -- 172.21.15.40:0/3200671639 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c0aa7b0 con 0x7f6b100b0c90 2024-04-22T05:50:44.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6b14e28640 1 -- 172.21.15.40:0/3200671639 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0aab80 con 0x7f6b100b0c90 2024-04-22T05:50:44.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6a5578e640 1 --2- 172.21.15.40:0/3715684112 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a4802c4f0 0x7f6a480c1db0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a50072930 tx=0x7f6a5009a260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ae98b0e89dcdba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6a54f8d640 1 -- 172.21.15.40:0/3715684112 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5012a440 con 0x7f6a4802c4f0 2024-04-22T05:50:44.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6b1808e640 1 -- 172.21.15.40:0/3200671639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b100b0c90 msgr2=0x7f6b1006ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6b1808e640 1 --2- 172.21.15.40:0/3200671639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b100b0c90 0x7f6b1006ff00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009870 tx=0x7f6b0c01c4d0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f9574aa4640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9568035d00 0x7f956802bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec2a0f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6eb40072c0 0x7f6eb40c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec2a0f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6eb402c9c0 0x7f6eb40d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec2a0f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb40076a0 con 0x7f6eb402d690 2024-04-22T05:50:44.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec2a0f640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb402c530 con 0x7f6eb402c9c0 2024-04-22T05:50:44.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec2a0f640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb402e9d0 con 0x7f6eb40072c0 2024-04-22T05:50:44.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec120c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6eb402d690 0x7f6eb40c2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56102/0 (socket says 172.21.15.40:56102) 2024-04-22T05:50:44.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec120c640 1 -- 172.21.15.40:0/2101839082 learned_addr learned my addr 172.21.15.40:0/2101839082 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:50:44.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec1a0d640 1 --2- 172.21.15.40:0/2101839082 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6eb40072c0 0x7f6eb40c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f9574aa4640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9568033f10 0x7f95680c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f9574aa4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95680360e0 con 0x7f95680c8310 2024-04-22T05:50:44.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6b1808e640 1 -- 172.21.15.40:0/3200671639 shutdown_connections 2024-04-22T05:50:44.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6b1808e640 1 --2- 172.21.15.40:0/3200671639 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b100afe00 0x7f6b10138890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6b1808e640 1 --2- 172.21.15.40:0/3200671639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b100b0c90 0x7f6b1006ff00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6b1808e640 1 -- 172.21.15.40:0/3200671639 >> 172.21.15.40:0/3200671639 conn(0x7f6b100b4cc0 msgr2=0x7f6b100b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f9574aa4640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956802d560 con 0x7f9568035d00 2024-04-22T05:50:44.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f9574aa4640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956802b9e0 con 0x7f9568033f10 2024-04-22T05:50:44.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f956e59c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9568035d00 0x7f956802bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f7d827fc640 1 -- 172.21.15.40:0/4205624072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d740a6660 con 0x7f7d840b4210 2024-04-22T05:50:44.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f7d827fc640 1 -- 172.21.15.40:0/4205624072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d740a67c0 con 0x7f7d840b4210 2024-04-22T05:50:44.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f7d827fc640 1 -- 172.21.15.40:0/4205624072 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d740a6b50 con 0x7f7d840b4210 2024-04-22T05:50:44.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f7d88fe1640 1 -- 172.21.15.40:0/4205624072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d840b4210 msgr2=0x7f7d840aeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f7d88fe1640 1 --2- 172.21.15.40:0/4205624072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d840b4210 0x7f7d840aeff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7d84141f60 tx=0x7f7d7401c680 comp rx=0 tx=0).stop 2024-04-22T05:50:44.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f956e59c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9568035d00 0x7f956802bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:58972/0 (socket says 172.21.15.40:58972) 2024-04-22T05:50:44.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f956e59c640 1 -- 172.21.15.40:0/2877944501 learned_addr learned my addr 172.21.15.40:0/2877944501 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f956dd9b640 1 --2- 172.21.15.40:0/2877944501 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9568033f10 0x7f95680c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec1a0d640 1 -- 172.21.15.40:0/2101839082 >> v2:172.21.15.62:3300/0 conn(0x7f6eb402c9c0 msgr2=0x7f6eb40d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:44.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec1a0d640 1 --2- 172.21.15.40:0/2101839082 >> v2:172.21.15.62:3300/0 conn(0x7f6eb402c9c0 0x7f6eb40d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec1a0d640 1 -- 172.21.15.40:0/2101839082 >> v1:172.21.15.40:6789/0 conn(0x7f6eb402d690 legacy=0x7f6eb40c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6ec1a0d640 1 -- 172.21.15.40:0/2101839082 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb402d240 con 0x7f6eb40072c0 2024-04-22T05:50:44.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1808e640 1 -- 172.21.15.40:0/3200671639 shutdown_connections 2024-04-22T05:50:44.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f956dd9b640 1 -- 172.21.15.40:0/2877944501 >> v2:172.21.15.62:3300/0 conn(0x7f9568035d00 msgr2=0x7f956802bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:44.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f956dd9b640 1 --2- 172.21.15.40:0/2877944501 >> v2:172.21.15.62:3300/0 conn(0x7f9568035d00 0x7f956802bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f956dd9b640 1 -- 172.21.15.40:0/2877944501 >> v1:172.21.15.40:6789/0 conn(0x7f95680c8310 legacy=0x7f95680c1950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:50:44.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f956dd9b640 1 -- 172.21.15.40:0/2877944501 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956802c460 con 0x7f9568033f10 2024-04-22T05:50:44.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6a54f8d640 1 -- 172.21.15.40:0/3715684112 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5012a5e0 con 0x7f6a4802c4f0 2024-04-22T05:50:44.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.158+0000 7f6a54f8d640 1 -- 172.21.15.40:0/3715684112 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5012aa10 con 0x7f6a4802c4f0 2024-04-22T05:50:44.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6a56f91640 1 -- 172.21.15.40:0/3715684112 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a4802c4f0 msgr2=0x7f6a480c1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6a56f91640 1 --2- 172.21.15.40:0/3715684112 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a4802c4f0 0x7f6a480c1db0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a50072930 tx=0x7f6a5009a260 comp rx=0 tx=0).stop 2024-04-22T05:50:44.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6a56f91640 1 -- 172.21.15.40:0/3715684112 shutdown_connections 2024-04-22T05:50:44.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6a56f91640 1 --2- 172.21.15.40:0/3715684112 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a4802c4f0 0x7f6a480c1db0 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a50072930 tx=0x7f6a5009a260 comp rx=0 tx=0).stop 2024-04-22T05:50:44.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6a56f91640 1 --2- 172.21.15.40:0/3715684112 >> v2:172.21.15.40:3300/0 conn(0x7f6a48037500 0x7f6a480c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6a56f91640 1 -- 172.21.15.40:0/3715684112 >> 172.21.15.40:0/3715684112 conn(0x7f6a4803a850 msgr2=0x7f6a48036b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 -- 172.21.15.40:0/4205624072 shutdown_connections 2024-04-22T05:50:44.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 --2- 172.21.15.40:0/4205624072 >> v2:172.21.15.62:3300/0 conn(0x7f7d8413e1f0 0x7f7d8413e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 --2- 172.21.15.40:0/4205624072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d840b4210 0x7f7d840aeff0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 -- 172.21.15.40:0/4205624072 >> 172.21.15.40:0/4205624072 conn(0x7f7d8406b730 msgr2=0x7f7d840b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1808e640 1 -- 172.21.15.40:0/3200671639 wait complete. 2024-04-22T05:50:44.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f66d77fe640 1 --2- 172.21.15.40:0/560357895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d80701a0 0x7f66d80ae320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f66d8073c50 tx=0x7f66c801ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adade5c730dbffdd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 -- 172.21.15.40:0/4205624072 shutdown_connections 2024-04-22T05:50:44.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1808e640 1 Processor -- start 2024-04-22T05:50:44.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f66d6ffd640 1 -- 172.21.15.40:0/560357895 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c80a89d0 con 0x7f66d80701a0 2024-04-22T05:50:44.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f66d6ffd640 1 -- 172.21.15.40:0/560357895 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c80a8b30 con 0x7f66d80701a0 2024-04-22T05:50:44.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f66d6ffd640 1 -- 172.21.15.40:0/560357895 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c80b0440 con 0x7f66d80701a0 2024-04-22T05:50:44.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 -- 172.21.15.40:0/4205624072 wait complete. 2024-04-22T05:50:44.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 Processor -- start 2024-04-22T05:50:44.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 -- start start 2024-04-22T05:50:44.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7d840ad1b0 0x7f7d8413eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d840b4210 0x7f7d841414f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8413e1f0 0x7f7d84141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d84141f70 con 0x7f7d840b4210 2024-04-22T05:50:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d84070a00 con 0x7f7d8413e1f0 2024-04-22T05:50:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d88fe1640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8413f1e0 con 0x7f7d840ad1b0 2024-04-22T05:50:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6ec1a0d640 1 --2- 172.21.15.40:0/2101839082 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6eb40072c0 0x7f6eb40c8e90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc072e20 tx=0x7f6ebc095de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42827e9700812778 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f66de70c640 1 -- 172.21.15.40:0/560357895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d80701a0 msgr2=0x7f66d80ae320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f66de70c640 1 --2- 172.21.15.40:0/560357895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d80701a0 0x7f66d80ae320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f66d8073c50 tx=0x7f66c801ed30 comp rx=0 tx=0).stop 2024-04-22T05:50:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1808e640 1 -- start start 2024-04-22T05:50:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1808e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b100afa20 0x7f6b10136ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1808e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b100afe00 0x7f6b101d9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1808e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b100b0c90 0x7f6b101da290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1808e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101daa30 con 0x7f6b100b0c90 2024-04-22T05:50:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1808e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b10130bd0 con 0x7f6b100afe00 2024-04-22T05:50:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1808e640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b10130d90 con 0x7f6b100afa20 2024-04-22T05:50:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1662b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b100b0c90 0x7f6b101da290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b15e2a640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b100afa20 0x7f6b10136ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1662b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b100b0c90 0x7f6b101da290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59740/0 (socket says 172.21.15.40:59740) 2024-04-22T05:50:44.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1662b640 1 -- 172.21.15.40:0/1059714771 learned_addr learned my addr 172.21.15.40:0/1059714771 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8413e1f0 0x7f7d84141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8413e1f0 0x7f7d84141a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:58984/0 (socket says 172.21.15.40:58984) 2024-04-22T05:50:44.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d83fff640 1 -- 172.21.15.40:0/3624805635 learned_addr learned my addr 172.21.15.40:0/3624805635 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f66de70c640 1 -- 172.21.15.40:0/560357895 shutdown_connections 2024-04-22T05:50:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f66de70c640 1 --2- 172.21.15.40:0/560357895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d80701a0 0x7f66d80ae320 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f66de70c640 1 --2- 172.21.15.40:0/560357895 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66d806f450 0x7f66d813af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f66de70c640 1 --2- 172.21.15.40:0/560357895 >> v2:172.21.15.40:3300/0 conn(0x7f66d806f070 0x7f66d813e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f66de70c640 1 -- 172.21.15.40:0/560357895 >> 172.21.15.40:0/560357895 conn(0x7f66d80b3920 msgr2=0x7f66d80b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1662b640 1 -- 172.21.15.40:0/1059714771 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b100afa20 msgr2=0x7f6b10136ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1662b640 1 --2- 172.21.15.40:0/1059714771 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b100afa20 0x7f6b10136ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1662b640 1 -- 172.21.15.40:0/1059714771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b100afe00 msgr2=0x7f6b101d9d50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T05:50:44.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f66de70c640 1 -- 172.21.15.40:0/560357895 shutdown_connections 2024-04-22T05:50:44.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1662b640 1 --2- 172.21.15.40:0/1059714771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b100afe00 0x7f6b101d9d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1662b640 1 -- 172.21.15.40:0/1059714771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c0094d0 con 0x7f6b100b0c90 2024-04-22T05:50:44.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6ec0a0b640 1 -- 172.21.15.40:0/2101839082 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc11eaa0 con 0x7f6eb40072c0 2024-04-22T05:50:44.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6ec0a0b640 1 -- 172.21.15.40:0/2101839082 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc11ec40 con 0x7f6eb40072c0 2024-04-22T05:50:44.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b15e2a640 1 --2- 172.21.15.40:0/1059714771 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b100afa20 0x7f6b10136ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:50:44.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6ec0a0b640 1 -- 172.21.15.40:0/2101839082 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc126640 con 0x7f6eb40072c0 2024-04-22T05:50:44.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d837fe640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7d840ad1b0 0x7f7d8413eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d83fff640 1 -- 172.21.15.40:0/3624805635 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7d840ad1b0 msgr2=0x7f7d8413eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d83fff640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7d840ad1b0 0x7f7d8413eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f7d83fff640 1 -- 172.21.15.40:0/3624805635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d840b4210 msgr2=0x7f7d841414f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T05:50:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f6b1662b640 1 --2- 172.21.15.40:0/1059714771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b100b0c90 0x7f6b101da290 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800efc0 tx=0x7f6b0800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6b067fc640 1 -- 172.21.15.40:0/1059714771 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08014480 con 0x7f6b100b0c90 2024-04-22T05:50:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a56f91640 1 -- 172.21.15.40:0/3715684112 shutdown_connections 2024-04-22T05:50:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6b067fc640 1 -- 172.21.15.40:0/1059714771 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0800f040 con 0x7f6b100b0c90 2024-04-22T05:50:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6b1808e640 1 -- 172.21.15.40:0/1059714771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b10131010 con 0x7f6b100b0c90 2024-04-22T05:50:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6b067fc640 1 -- 172.21.15.40:0/1059714771 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08014840 con 0x7f6b100b0c90 2024-04-22T05:50:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6b1808e640 1 -- 172.21.15.40:0/1059714771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b101314e0 con 0x7f6b100b0c90 2024-04-22T05:50:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/2101839082 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6eb40072c0 msgr2=0x7f6eb40c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6ec2a0f640 1 --2- 172.21.15.40:0/2101839082 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6eb40072c0 0x7f6eb40c8e90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc072e20 tx=0x7f6ebc095de0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f956dd9b640 1 --2- 172.21.15.40:0/2877944501 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9568033f10 0x7f95680c7dd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95640097a0 tx=0x7f956401c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0af2249112efcb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.162+0000 7f66de70c640 1 -- 172.21.15.40:0/560357895 wait complete. 2024-04-22T05:50:44.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66de70c640 1 Processor -- start 2024-04-22T05:50:44.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f956d59a640 1 -- 172.21.15.40:0/2877944501 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640ab600 con 0x7f9568033f10 2024-04-22T05:50:44.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f956d59a640 1 -- 172.21.15.40:0/2877944501 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95640a7050 con 0x7f9568033f10 2024-04-22T05:50:44.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f956d59a640 1 -- 172.21.15.40:0/2877944501 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640ab980 con 0x7f9568033f10 2024-04-22T05:50:44.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6b1808e640 1 -- 172.21.15.40:0/1059714771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b101dab70 con 0x7f6b100b0c90 2024-04-22T05:50:44.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6b067fc640 1 -- 172.21.15.40:0/1059714771 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b080149a0 con 0x7f6b100b0c90 2024-04-22T05:50:44.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a56f91640 1 -- 172.21.15.40:0/3715684112 wait complete. 2024-04-22T05:50:44.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a56f91640 1 Processor -- start 2024-04-22T05:50:44.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f9574aa4640 1 -- 172.21.15.40:0/2877944501 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9568033f10 msgr2=0x7f95680c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f9574aa4640 1 --2- 172.21.15.40:0/2877944501 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9568033f10 0x7f95680c7dd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95640097a0 tx=0x7f956401c390 comp rx=0 tx=0).stop 2024-04-22T05:50:44.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f9574aa4640 1 -- 172.21.15.40:0/2877944501 shutdown_connections 2024-04-22T05:50:44.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f9574aa4640 1 --2- 172.21.15.40:0/2877944501 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9568033f10 0x7f95680c7dd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f9574aa4640 1 --2- 172.21.15.40:0/2877944501 >> v2:172.21.15.62:3300/0 conn(0x7f9568035d00 0x7f956802bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f9574aa4640 1 -- 172.21.15.40:0/2877944501 >> 172.21.15.40:0/2877944501 conn(0x7f95680071a0 msgr2=0x7f95680075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f9574aa4640 1 -- 172.21.15.40:0/2877944501 shutdown_connections 2024-04-22T05:50:44.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f9574aa4640 1 -- 172.21.15.40:0/2877944501 wait complete. 2024-04-22T05:50:44.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6b067fc640 1 --2- 172.21.15.40:0/1059714771 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6af406aaa0 0x7f6af406cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66de70c640 1 -- start start 2024-04-22T05:50:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66de70c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66d806f070 0x7f66d81e6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66de70c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d806f450 0x7f66d813ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66de70c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d80701a0 0x7f66d813f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66de70c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d81e67d0 con 0x7f66d80701a0 2024-04-22T05:50:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66de70c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d81e6f30 con 0x7f66d806f450 2024-04-22T05:50:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66de70c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d813a6c0 con 0x7f66d806f070 2024-04-22T05:50:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66d7fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66d806f070 0x7f66d81e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66d7fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66d806f070 0x7f66d81e6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35284/0 (socket says 172.21.15.40:35284) 2024-04-22T05:50:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66d7fff640 1 -- 172.21.15.40:0/611592521 learned_addr learned my addr 172.21.15.40:0/611592521 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66d77fe640 1 --2- 172.21.15.40:0/611592521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d806f450 0x7f66d813ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6b15e2a640 1 --2- 172.21.15.40:0/1059714771 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6af406aaa0 0x7f6af406cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66d7fff640 1 -- 172.21.15.40:0/611592521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d806f450 msgr2=0x7f66d813ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66d7fff640 1 --2- 172.21.15.40:0/611592521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d806f450 0x7f66d813ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66d7fff640 1 -- 172.21.15.40:0/611592521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d80701a0 msgr2=0x7f66d813f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66d7fff640 1 --2- 172.21.15.40:0/611592521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d80701a0 0x7f66d813f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f66d7fff640 1 -- 172.21.15.40:0/611592521 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c80094d0 con 0x7f66d806f070 2024-04-22T05:50:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a56f91640 1 -- start start 2024-04-22T05:50:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a56f91640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a48037500 0x7f6a48160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a56f91640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a480c7dc0 0x7f6a480c2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a56f91640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a480c2b80 0x7f6a480c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a56f91640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c9440 con 0x7f6a480c7dc0 2024-04-22T05:50:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a56f91640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c4360 con 0x7f6a480c2b80 2024-04-22T05:50:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a56f91640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c8f10 con 0x7f6a48037500 2024-04-22T05:50:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a55f8f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a48037500 0x7f6a48160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a55f8f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a48037500 0x7f6a48160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35290/0 (socket says 172.21.15.40:35290) 2024-04-22T05:50:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a55f8f640 1 -- 172.21.15.40:0/3292179992 learned_addr learned my addr 172.21.15.40:0/3292179992 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a56790640 1 --2- 172.21.15.40:0/3292179992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a480c2b80 0x7f6a480c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6b15e2a640 1 --2- 172.21.15.40:0/1059714771 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6af406aaa0 0x7f6af406cf60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6b00007fd0 tx=0x7f6b00009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f9574aa4640 1 Processor -- start 2024-04-22T05:50:44.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f66d7fff640 1 --2- 172.21.15.40:0/611592521 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66d806f070 0x7f66d81e6290 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00b410 tx=0x7f66cc00b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a55f8f640 1 -- 172.21.15.40:0/3292179992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a480c2b80 msgr2=0x7f6a480c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a55f8f640 1 --2- 172.21.15.40:0/3292179992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a480c2b80 0x7f6a480c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a55f8f640 1 -- 172.21.15.40:0/3292179992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a480c7dc0 msgr2=0x7f6a480c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f6a55f8f640 1 --2- 172.21.15.40:0/3292179992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a480c7dc0 0x7f6a480c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6a55f8f640 1 -- 172.21.15.40:0/3292179992 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5004f100 con 0x7f6a48037500 2024-04-22T05:50:44.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.166+0000 7f9574aa4640 1 -- start start 2024-04-22T05:50:44.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f66d4ff9640 1 -- 172.21.15.40:0/611592521 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc011830 con 0x7f66d806f070 2024-04-22T05:50:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f66d4ff9640 1 -- 172.21.15.40:0/611592521 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc011990 con 0x7f66d806f070 2024-04-22T05:50:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f66d4ff9640 1 -- 172.21.15.40:0/611592521 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc011d40 con 0x7f66d806f070 2024-04-22T05:50:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6a55f8f640 1 --2- 172.21.15.40:0/3292179992 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a48037500 0x7f6a48160670 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000d9c0 tx=0x7f6a4000ddc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f9574aa4640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9568033f10 0x7f95680c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f9574aa4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9568035d00 0x7f95680c2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f9574aa4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95680c8310 0x7f95680c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f9574aa4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95680c5a40 con 0x7f9568035d00 2024-04-22T05:50:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f9574aa4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95680c55b0 con 0x7f95680c8310 2024-04-22T05:50:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f9574aa4640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95680c3090 con 0x7f9568033f10 2024-04-22T05:50:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f956ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95680c8310 0x7f95680c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f956ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95680c8310 0x7f95680c2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:59014/0 (socket says 172.21.15.40:59014) 2024-04-22T05:50:44.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f956ed9d640 1 -- 172.21.15.40:0/3014846278 learned_addr learned my addr 172.21.15.40:0/3014846278 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f66de70c640 1 -- 172.21.15.40:0/611592521 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d8071b20 con 0x7f66d806f070 2024-04-22T05:50:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f66de70c640 1 -- 172.21.15.40:0/611592521 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d813f9f0 con 0x7f66d806f070 2024-04-22T05:50:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f956e59c640 1 --2- 172.21.15.40:0/3014846278 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9568033f10 0x7f95680c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f66de70c640 1 -- 172.21.15.40:0/611592521 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d81e6c50 con 0x7f66d806f070 2024-04-22T05:50:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f956e59c640 1 -- 172.21.15.40:0/3014846278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95680c8310 msgr2=0x7f95680c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f956e59c640 1 --2- 172.21.15.40:0/3014846278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95680c8310 0x7f95680c2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f956e59c640 1 -- 172.21.15.40:0/3014846278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9568035d00 msgr2=0x7f95680c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f956e59c640 1 --2- 172.21.15.40:0/3014846278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9568035d00 0x7f95680c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f956e59c640 1 -- 172.21.15.40:0/3014846278 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95640093b0 con 0x7f9568033f10 2024-04-22T05:50:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f956e59c640 1 --2- 172.21.15.40:0/3014846278 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9568033f10 0x7f95680c73e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f957004fb10 tx=0x7f9570067200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6a467fc640 1 -- 172.21.15.40:0/3292179992 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40007bd0 con 0x7f6a48037500 2024-04-22T05:50:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6a56f91640 1 -- 172.21.15.40:0/3292179992 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a480c5330 con 0x7f6a48037500 2024-04-22T05:50:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6a56f91640 1 -- 172.21.15.40:0/3292179992 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a480c4a50 con 0x7f6a48037500 2024-04-22T05:50:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6a467fc640 1 -- 172.21.15.40:0/3292179992 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a40007d30 con 0x7f6a48037500 2024-04-22T05:50:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f7d82ffd640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d840b4210 0x7f7d841414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f7d83fff640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d840b4210 0x7f7d841414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f7d83fff640 1 -- 172.21.15.40:0/3624805635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d740094d0 con 0x7f7d8413e1f0 2024-04-22T05:50:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f7d82ffd640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d840b4210 0x7f7d841414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:50:44.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/2101839082 shutdown_connections 2024-04-22T05:50:44.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6ec2a0f640 1 --2- 172.21.15.40:0/2101839082 >> v2:172.21.15.62:3300/0 conn(0x7f6eb402c9c0 0x7f6eb40d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f7d83fff640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8413e1f0 0x7f7d84141a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c001570 tx=0x7f7d7c00cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f7d6bfff640 1 -- 172.21.15.40:0/3624805635 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c003db0 con 0x7f7d8413e1f0 2024-04-22T05:50:44.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f7d6bfff640 1 -- 172.21.15.40:0/3624805635 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7c005070 con 0x7f7d8413e1f0 2024-04-22T05:50:44.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f7d6bfff640 1 -- 172.21.15.40:0/3624805635 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c005420 con 0x7f7d8413e1f0 2024-04-22T05:50:44.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6ec2a0f640 1 --2- 172.21.15.40:0/2101839082 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6eb40072c0 0x7f6eb40c8e90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/2101839082 >> 172.21.15.40:0/2101839082 conn(0x7f6eb4006290 msgr2=0x7f6eb40066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/2101839082 shutdown_connections 2024-04-22T05:50:44.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6a467fc640 1 -- 172.21.15.40:0/3292179992 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40007ea0 con 0x7f6a48037500 2024-04-22T05:50:44.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/2101839082 wait complete. 2024-04-22T05:50:44.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f66d4ff9640 1 -- 172.21.15.40:0/611592521 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f66cc01a400 con 0x7f66d806f070 2024-04-22T05:50:44.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f66d4ff9640 1 --2- 172.21.15.40:0/611592521 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f66c4066760 0x7f66c4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f66d4ff9640 1 -- 172.21.15.40:0/611592521 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f66cc083f70 con 0x7f66d806f070 2024-04-22T05:50:44.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f66d77fe640 1 --2- 172.21.15.40:0/611592521 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f66c4066760 0x7f66c4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.170+0000 7f6a56f91640 1 -- 172.21.15.40:0/3292179992 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a38005050 con 0x7f6a48037500 2024-04-22T05:50:44.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f7d88fe1640 1 -- 172.21.15.40:0/3624805635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d84145050 con 0x7f7d8413e1f0 2024-04-22T05:50:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f66d77fe640 1 --2- 172.21.15.40:0/611592521 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f66c4066760 0x7f66c4068c20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66c80b2000 tx=0x7f66c80aa070 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f7d88fe1640 1 -- 172.21.15.40:0/3624805635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8413f780 con 0x7f7d8413e1f0 2024-04-22T05:50:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f7d88fe1640 1 -- 172.21.15.40:0/3624805635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d8413fa00 con 0x7f7d8413e1f0 2024-04-22T05:50:44.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec2a0f640 1 Processor -- start 2024-04-22T05:50:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6b067fc640 1 -- 172.21.15.40:0/1059714771 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6b080559c0 con 0x7f6b100b0c90 2024-04-22T05:50:44.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6b067fc640 1 -- 172.21.15.40:0/1059714771 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f6b101dab70 con 0x7f6b100b0c90 2024-04-22T05:50:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec2a0f640 1 -- start start 2024-04-22T05:50:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec2a0f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6eb40072c0 0x7f6eb40c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec2a0f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eb402c9c0 0x7f6eb40c5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec2a0f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb40c81b0 0x7f6eb40c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec2a0f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb40d2f00 con 0x7f6eb402c9c0 2024-04-22T05:50:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec2a0f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb40c3860 con 0x7f6eb40c81b0 2024-04-22T05:50:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec2a0f640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb40c8b70 con 0x7f6eb40072c0 2024-04-22T05:50:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec1a0d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6eb40072c0 0x7f6eb40c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec1a0d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6eb40072c0 0x7f6eb40c4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35308/0 (socket says 172.21.15.40:35308) 2024-04-22T05:50:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec1a0d640 1 -- 172.21.15.40:0/382785431 learned_addr learned my addr 172.21.15.40:0/382785431 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec220e640 1 --2- 172.21.15.40:0/382785431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb40c81b0 0x7f6eb40c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec220e640 1 -- 172.21.15.40:0/382785431 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6eb40072c0 msgr2=0x7f6eb40c4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec220e640 1 --2- 172.21.15.40:0/382785431 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6eb40072c0 0x7f6eb40c4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec220e640 1 -- 172.21.15.40:0/382785431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eb402c9c0 msgr2=0x7f6eb40c5430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T05:50:44.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec220e640 1 --2- 172.21.15.40:0/382785431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eb402c9c0 0x7f6eb40c5430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec220e640 1 -- 172.21.15.40:0/382785431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc04f100 con 0x7f6eb40c81b0 2024-04-22T05:50:44.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.174+0000 7f6ec220e640 1 --2- 172.21.15.40:0/382785431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb40c81b0 0x7f6eb40c8630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6eb402cdd0 tx=0x7f6eac00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f7d6bfff640 1 -- 172.21.15.40:0/3624805635 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7d7c005580 con 0x7f7d8413e1f0 2024-04-22T05:50:44.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f955effd640 1 -- 172.21.15.40:0/3014846278 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570071e30 con 0x7f9568033f10 2024-04-22T05:50:44.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f7d6bfff640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f7d70066690 0x7f7d70068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f7d6bfff640 1 -- 172.21.15.40:0/3624805635 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7d7c083cb0 con 0x7f7d8413e1f0 2024-04-22T05:50:44.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f7d837fe640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f7d70066690 0x7f7d70068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f9574aa4640 1 -- 172.21.15.40:0/3014846278 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9568039d30 con 0x7f9568033f10 2024-04-22T05:50:44.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f9574aa4640 1 -- 172.21.15.40:0/3014846278 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95680c39e0 con 0x7f9568033f10 2024-04-22T05:50:44.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f9574aa4640 1 -- 172.21.15.40:0/3014846278 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9568039f20 con 0x7f9568033f10 2024-04-22T05:50:44.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f7d837fe640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f7d70066690 0x7f7d70068b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d740af000 tx=0x7f7d74008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f955effd640 1 -- 172.21.15.40:0/3014846278 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957009b440 con 0x7f9568033f10 2024-04-22T05:50:44.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f955effd640 1 -- 172.21.15.40:0/3014846278 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957009b7b0 con 0x7f9568033f10 2024-04-22T05:50:44.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f6a467fc640 1 -- 172.21.15.40:0/3292179992 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6a40010670 con 0x7f6a48037500 2024-04-22T05:50:44.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f6a467fc640 1 --2- 172.21.15.40:0/3292179992 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6a34066760 0x7f6a34068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f6a467fc640 1 -- 172.21.15.40:0/3292179992 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6a400846b0 con 0x7f6a48037500 2024-04-22T05:50:44.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.178+0000 7f6a5578e640 1 --2- 172.21.15.40:0/3292179992 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6a34066760 0x7f6a34068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.182+0000 7f66d4ff9640 1 -- 172.21.15.40:0/611592521 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f66cc051540 con 0x7f66d806f070 2024-04-22T05:50:44.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.182+0000 7f6a5578e640 1 --2- 172.21.15.40:0/3292179992 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6a34066760 0x7f6a34068c20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a4802c1e0 tx=0x7f6a5012c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.182+0000 7f6a467fc640 1 -- 172.21.15.40:0/3292179992 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f6a40051d30 con 0x7f6a48037500 2024-04-22T05:50:44.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.182+0000 7f955effd640 1 -- 172.21.15.40:0/3014846278 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f957009b910 con 0x7f9568033f10 2024-04-22T05:50:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.186+0000 7f955effd640 1 --2- 172.21.15.40:0/3014846278 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9550066690 0x7f9550068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.186+0000 7f955effd640 1 -- 172.21.15.40:0/3014846278 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f95700fe060 con 0x7f9568033f10 2024-04-22T05:50:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.186+0000 7f956dd9b640 1 --2- 172.21.15.40:0/3014846278 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9550066690 0x7f9550068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.186+0000 7f956dd9b640 1 --2- 172.21.15.40:0/3014846278 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9550066690 0x7f9550068b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f95640ad000 tx=0x7f9564004740 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.186+0000 7f6eb27fc640 1 -- 172.21.15.40:0/382785431 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac005a20 con 0x7f6eb40c81b0 2024-04-22T05:50:44.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.186+0000 7f6eb27fc640 1 -- 172.21.15.40:0/382785431 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac005b80 con 0x7f6eb40c81b0 2024-04-22T05:50:44.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.186+0000 7f6eb27fc640 1 -- 172.21.15.40:0/382785431 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac011630 con 0x7f6eb40c81b0 2024-04-22T05:50:44.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.186+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/382785431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb40c4380 con 0x7f6eb40c81b0 2024-04-22T05:50:44.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.186+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/382785431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb40d3560 con 0x7f6eb40c81b0 2024-04-22T05:50:44.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.186+0000 7f7d6bfff640 1 -- 172.21.15.40:0/3624805635 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f7d7c051280 con 0x7f7d8413e1f0 2024-04-22T05:50:44.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.190+0000 7f6eb27fc640 1 -- 172.21.15.40:0/382785431 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6eac011790 con 0x7f6eb40c81b0 2024-04-22T05:50:44.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.190+0000 7f6eb27fc640 1 --2- 172.21.15.40:0/382785431 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6ea0066690 0x7f6ea0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.190+0000 7f6eb27fc640 1 -- 172.21.15.40:0/382785431 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6eac083b90 con 0x7f6eb40c81b0 2024-04-22T05:50:44.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.190+0000 7f955effd640 1 -- 172.21.15.40:0/3014846278 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f95700cb630 con 0x7f9568033f10 2024-04-22T05:50:44.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.190+0000 7f6ec1a0d640 1 --2- 172.21.15.40:0/382785431 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6ea0066690 0x7f6ea0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.194+0000 7f6ec1a0d640 1 --2- 172.21.15.40:0/382785431 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6ea0066690 0x7f6ea0068b50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc063ce0 tx=0x7f6ebc06c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.194+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/382785431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb40c6c70 con 0x7f6eb40c81b0 2024-04-22T05:50:44.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f3628959640 1 Processor -- start 2024-04-22T05:50:44.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f3628959640 1 -- start start 2024-04-22T05:50:44.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f3628959640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36240adb80 0x7f3624141410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f3628959640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f36240adf60 0x7f3624141950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f3628959640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3624141e90 con 0x7f3624144430 2024-04-22T05:50:44.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f3628959640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362406e950 con 0x7f36240adb80 2024-04-22T05:50:44.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f3628959640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3624140fc0 con 0x7f36240adf60 2024-04-22T05:50:44.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f361bfff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f36240adf60 0x7f3624141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f361bfff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f36240adf60 0x7f3624141950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35314/0 (socket says 172.21.15.40:35314) 2024-04-22T05:50:44.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f361bfff640 1 -- 172.21.15.40:0/1328454320 learned_addr learned my addr 172.21.15.40:0/1328454320 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f362259c640 1 --2- 172.21.15.40:0/1328454320 >> v2:172.21.15.40:3300/0 conn(0x7f36240adb80 0x7f3624141410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f362259c640 1 -- 172.21.15.40:0/1328454320 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f36240adf60 msgr2=0x7f3624141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:44.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f362259c640 1 --2- 172.21.15.40:0/1328454320 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f36240adf60 0x7f3624141950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f362259c640 1 -- 172.21.15.40:0/1328454320 >> v1:172.21.15.62:6789/0 conn(0x7f3624144430 legacy=0x7f36241448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.198+0000 7f362259c640 1 -- 172.21.15.40:0/1328454320 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362406eb30 con 0x7f36240adb80 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.202+0000 7f362259c640 1 --2- 172.21.15.40:0/1328454320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240adb80 0x7f3624141410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f360c009a50 tx=0x7f360c01c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62d1625bb63b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.202+0000 7f3621d9b640 1 -- 172.21.15.40:0/1328454320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c0a6620 con 0x7f36240adb80 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.202+0000 7f3621d9b640 1 -- 172.21.15.40:0/1328454320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c0a6780 con 0x7f36240adb80 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.202+0000 7f3621d9b640 1 -- 172.21.15.40:0/1328454320 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c0a6620 con 0x7f36240adb80 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.202+0000 7f3628959640 1 -- 172.21.15.40:0/1328454320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240adb80 msgr2=0x7f3624141410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.202+0000 7f3628959640 1 --2- 172.21.15.40:0/1328454320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240adb80 0x7f3624141410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f360c009a50 tx=0x7f360c01c6d0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.202+0000 7f3628959640 1 -- 172.21.15.40:0/1328454320 shutdown_connections 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.202+0000 7f3628959640 1 --2- 172.21.15.40:0/1328454320 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f36240adf60 0x7f3624141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.202+0000 7f3628959640 1 --2- 172.21.15.40:0/1328454320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240adb80 0x7f3624141410 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f360c009a50 tx=0x7f360c01c6d0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.202+0000 7f3628959640 1 -- 172.21.15.40:0/1328454320 >> 172.21.15.40:0/1328454320 conn(0x7f36240b28a0 msgr2=0x7f36240b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.202+0000 7f3628959640 1 -- 172.21.15.40:0/1328454320 shutdown_connections 2024-04-22T05:50:44.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.202+0000 7f3628959640 1 -- 172.21.15.40:0/1328454320 wait complete. 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f3628959640 1 Processor -- start 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f3628959640 1 -- start start 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f3628959640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240adf60 0x7f3624134cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f3628959640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3624144430 0x7f362406f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f3628959640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f362406faa0 0x7f36241e0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f3628959640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36241362d0 con 0x7f36240adf60 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f3628959640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36241365b0 con 0x7f362406faa0 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f3628959640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3624070ff0 con 0x7f3624144430 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f362259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240adf60 0x7f3624134cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f362259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240adf60 0x7f3624134cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59782/0 (socket says 172.21.15.40:59782) 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f362259c640 1 -- 172.21.15.40:0/2043012521 learned_addr learned my addr 172.21.15.40:0/2043012521 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f362259c640 1 -- 172.21.15.40:0/2043012521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240adf60 msgr2=0x7f3624134cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send send error: (32) Broken pipe 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f362259c640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240adf60 0x7f3624134cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).write auth request write failed r=-32 ((32) Broken pipe) 2024-04-22T05:50:44.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f362259c640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240adf60 0x7f3624134cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T05:50:44.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f3622d9d640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f362406faa0 0x7f36241e0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f361bfff640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3624144430 0x7f362406f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.206+0000 7f3622d9d640 1 -- 172.21.15.40:0/2043012521 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3624144430 msgr2=0x7f362406f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.210+0000 7f6eb27fc640 1 -- 172.21.15.40:0/382785431 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f6eac015080 con 0x7f6eb40c81b0 2024-04-22T05:50:44.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.210+0000 7f3622d9d640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3624144430 0x7f362406f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.210+0000 7f3622d9d640 1 -- 172.21.15.40:0/2043012521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240adf60 msgr2=0x7f3624134cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T05:50:44.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.210+0000 7f3622d9d640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240adf60 0x7f3624134cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.210+0000 7f3622d9d640 1 -- 172.21.15.40:0/2043012521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c009420 con 0x7f362406faa0 2024-04-22T05:50:44.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.210+0000 7f3622d9d640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f362406faa0 0x7f36241e0850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c730 tx=0x7f361c00cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.210+0000 7f361bfff640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3624144430 0x7f362406f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:50:44.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.222+0000 7f361affd640 1 -- 172.21.15.40:0/2043012521 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c006d60 con 0x7f362406faa0 2024-04-22T05:50:44.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.222+0000 7f361affd640 1 -- 172.21.15.40:0/2043012521 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c003980 con 0x7f362406faa0 2024-04-22T05:50:44.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.222+0000 7f361affd640 1 -- 172.21.15.40:0/2043012521 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c003d70 con 0x7f362406faa0 2024-04-22T05:50:44.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.222+0000 7f3628959640 1 -- 172.21.15.40:0/2043012521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36241340d0 con 0x7f362406faa0 2024-04-22T05:50:44.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.222+0000 7f3628959640 1 -- 172.21.15.40:0/2043012521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3624135400 con 0x7f362406faa0 2024-04-22T05:50:44.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.226+0000 7f361affd640 1 -- 172.21.15.40:0/2043012521 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f361c0050d0 con 0x7f362406faa0 2024-04-22T05:50:44.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.226+0000 7f361affd640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f3604066760 0x7f3604068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.226+0000 7f361affd640 1 -- 172.21.15.40:0/2043012521 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f361c0839a0 con 0x7f362406faa0 2024-04-22T05:50:44.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.226+0000 7f3628959640 1 -- 172.21.15.40:0/2043012521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362406f280 con 0x7f362406faa0 2024-04-22T05:50:44.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.226+0000 7f362259c640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f3604066760 0x7f3604068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.226+0000 7f362259c640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f3604066760 0x7f3604068c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f360c01c710 tx=0x7f360c0a7000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.238+0000 7f361affd640 1 -- 172.21.15.40:0/2043012521 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f362406f280 con 0x7f362406faa0 2024-04-22T05:50:44.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.450+0000 7f06b7828640 1 -- 172.21.15.40:0/1378899348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f06b0072300 con 0x7f06b0071d60 2024-04-22T05:50:44.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.450+0000 7f06a5ffb640 1 -- 172.21.15.40:0/1378899348 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f06a00eb450 con 0x7f06b0071d60 2024-04-22T05:50:44.455 INFO:teuthology.orchestra.run.smithi040.stdout:60129542147 2024-04-22T05:50:44.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.454+0000 7f069b7fe640 1 -- 172.21.15.40:0/1378899348 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f06940667e0 msgr2=0x7f0694068ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.454+0000 7f069b7fe640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f06940667e0 0x7f0694068ca0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f06a8004e10 tx=0x7f06a8007500 comp rx=0 tx=0).stop 2024-04-22T05:50:44.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.454+0000 7f069b7fe640 1 -- 172.21.15.40:0/1378899348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b0071d60 msgr2=0x7f06b0131840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.454+0000 7f069b7fe640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b0071d60 0x7f06b0131840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f06a001c080 tx=0x7f06a001b300 comp rx=0 tx=0).stop 2024-04-22T05:50:44.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.454+0000 7f069b7fe640 1 -- 172.21.15.40:0/1378899348 shutdown_connections 2024-04-22T05:50:44.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.454+0000 7f069b7fe640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f06940667e0 0x7f0694068ca0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.454+0000 7f069b7fe640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00716b0 0x7f06b0136630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.454+0000 7f069b7fe640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b0071d60 0x7f06b0131840 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.454+0000 7f069b7fe640 1 --2- 172.21.15.40:0/1378899348 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f06b006fcf0 0x7f06b00abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.454+0000 7f069b7fe640 1 -- 172.21.15.40:0/1378899348 >> 172.21.15.40:0/1378899348 conn(0x7f06b00b27a0 msgr2=0x7f06b01351b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.454+0000 7f069b7fe640 1 -- 172.21.15.40:0/1378899348 shutdown_connections 2024-04-22T05:50:44.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.454+0000 7f069b7fe640 1 -- 172.21.15.40:0/1378899348 wait complete. 2024-04-22T05:50:44.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.458+0000 7f6b1808e640 1 -- 172.21.15.40:0/1059714771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6b101dab70 con 0x7f6b100b0c90 2024-04-22T05:50:44.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.458+0000 7f6b067fc640 1 -- 172.21.15.40:0/1059714771 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6b101dab70 con 0x7f6b100b0c90 2024-04-22T05:50:44.461 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-22T05:50:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.466+0000 7f6b1808e640 1 -- 172.21.15.40:0/1059714771 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6af406aaa0 msgr2=0x7f6af406cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.466+0000 7f6b1808e640 1 --2- 172.21.15.40:0/1059714771 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6af406aaa0 0x7f6af406cf60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6b00007fd0 tx=0x7f6b00009040 comp rx=0 tx=0).stop 2024-04-22T05:50:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.466+0000 7f6b1808e640 1 -- 172.21.15.40:0/1059714771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b100b0c90 msgr2=0x7f6b101da290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.466+0000 7f6b1808e640 1 --2- 172.21.15.40:0/1059714771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b100b0c90 0x7f6b101da290 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800efc0 tx=0x7f6b0800c490 comp rx=0 tx=0).stop 2024-04-22T05:50:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.466+0000 7f6b1808e640 1 -- 172.21.15.40:0/1059714771 shutdown_connections 2024-04-22T05:50:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.466+0000 7f6b1808e640 1 --2- 172.21.15.40:0/1059714771 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6af406aaa0 0x7f6af406cf60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.466+0000 7f6b1808e640 1 --2- 172.21.15.40:0/1059714771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b100afe00 0x7f6b101d9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.466+0000 7f6b1808e640 1 --2- 172.21.15.40:0/1059714771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b100b0c90 0x7f6b101da290 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.466+0000 7f6b1808e640 1 --2- 172.21.15.40:0/1059714771 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6b100afa20 0x7f6b10136ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.466+0000 7f6b1808e640 1 -- 172.21.15.40:0/1059714771 >> 172.21.15.40:0/1059714771 conn(0x7f6b100b4cc0 msgr2=0x7f6b10138590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.466+0000 7f6b1808e640 1 -- 172.21.15.40:0/1059714771 shutdown_connections 2024-04-22T05:50:44.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.466+0000 7f6b1808e640 1 -- 172.21.15.40:0/1059714771 wait complete. 2024-04-22T05:50:44.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.0 2024-04-22T05:50:44.481 DEBUG:teuthology.parallel:result is None 2024-04-22T05:50:44.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.482+0000 7f66de70c640 1 -- 172.21.15.40:0/611592521 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f66d8066d70 con 0x7f66d806f070 2024-04-22T05:50:44.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.486+0000 7f66d4ff9640 1 -- 172.21.15.40:0/611592521 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f66cc015090 con 0x7f66d806f070 2024-04-22T05:50:44.487 INFO:teuthology.orchestra.run.smithi040.stdout:60129542148 2024-04-22T05:50:44.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2024-04-22T05:50:44.489 DEBUG:teuthology.parallel:result is None 2024-04-22T05:50:44.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.490+0000 7f66de70c640 1 -- 172.21.15.40:0/611592521 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f66c4066760 msgr2=0x7f66c4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.490+0000 7f66de70c640 1 --2- 172.21.15.40:0/611592521 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f66c4066760 0x7f66c4068c20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66c80b2000 tx=0x7f66c80aa070 comp rx=0 tx=0).stop 2024-04-22T05:50:44.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.490+0000 7f66de70c640 1 -- 172.21.15.40:0/611592521 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66d806f070 msgr2=0x7f66d81e6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.490+0000 7f66de70c640 1 --2- 172.21.15.40:0/611592521 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66d806f070 0x7f66d81e6290 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00b410 tx=0x7f66cc00b8d0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.490+0000 7f66de70c640 1 -- 172.21.15.40:0/611592521 shutdown_connections 2024-04-22T05:50:44.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.490+0000 7f66de70c640 1 --2- 172.21.15.40:0/611592521 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f66c4066760 0x7f66c4068c20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.490+0000 7f66de70c640 1 --2- 172.21.15.40:0/611592521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d806f450 0x7f66d813ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.490+0000 7f66de70c640 1 --2- 172.21.15.40:0/611592521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d80701a0 0x7f66d813f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.490+0000 7f66de70c640 1 --2- 172.21.15.40:0/611592521 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66d806f070 0x7f66d81e6290 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.490+0000 7f66de70c640 1 -- 172.21.15.40:0/611592521 >> 172.21.15.40:0/611592521 conn(0x7f66d80b3920 msgr2=0x7f66d81419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.490+0000 7f66de70c640 1 -- 172.21.15.40:0/611592521 shutdown_connections 2024-04-22T05:50:44.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.490+0000 7f66de70c640 1 -- 172.21.15.40:0/611592521 wait complete. 2024-04-22T05:50:44.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.4 2024-04-22T05:50:44.511 DEBUG:teuthology.parallel:result is None 2024-04-22T05:50:44.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.510+0000 7f6a56f91640 1 -- 172.21.15.40:0/3292179992 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6a380055d0 con 0x7f6a48037500 2024-04-22T05:50:44.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f6a467fc640 1 -- 172.21.15.40:0/3292179992 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6a4001f090 con 0x7f6a48037500 2024-04-22T05:50:44.514 INFO:teuthology.orchestra.run.smithi040.stdout:55834574850 2024-04-22T05:50:44.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f9574aa4640 1 -- 172.21.15.40:0/3014846278 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f95680063d0 con 0x7f9568033f10 2024-04-22T05:50:44.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f955effd640 1 -- 172.21.15.40:0/3014846278 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f95700cec50 con 0x7f9568033f10 2024-04-22T05:50:44.515 INFO:teuthology.orchestra.run.smithi040.stdout:60129542146 2024-04-22T05:50:44.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f9574aa4640 1 -- 172.21.15.40:0/3014846278 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9550066690 msgr2=0x7f9550068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f9574aa4640 1 --2- 172.21.15.40:0/3014846278 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9550066690 0x7f9550068b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f95640ad000 tx=0x7f9564004740 comp rx=0 tx=0).stop 2024-04-22T05:50:44.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f9574aa4640 1 -- 172.21.15.40:0/3014846278 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9568033f10 msgr2=0x7f95680c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f9574aa4640 1 --2- 172.21.15.40:0/3014846278 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9568033f10 0x7f95680c73e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f957004fb10 tx=0x7f9570067200 comp rx=0 tx=0).stop 2024-04-22T05:50:44.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f9574aa4640 1 -- 172.21.15.40:0/3014846278 shutdown_connections 2024-04-22T05:50:44.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f9574aa4640 1 --2- 172.21.15.40:0/3014846278 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9550066690 0x7f9550068b50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f9574aa4640 1 --2- 172.21.15.40:0/3014846278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95680c8310 0x7f95680c2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f9574aa4640 1 --2- 172.21.15.40:0/3014846278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9568035d00 0x7f95680c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f9574aa4640 1 --2- 172.21.15.40:0/3014846278 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9568033f10 0x7f95680c73e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f9574aa4640 1 -- 172.21.15.40:0/3014846278 >> 172.21.15.40:0/3014846278 conn(0x7f95680071a0 msgr2=0x7f9568033320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f9574aa4640 1 -- 172.21.15.40:0/3014846278 shutdown_connections 2024-04-22T05:50:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f9574aa4640 1 -- 172.21.15.40:0/3014846278 wait complete. 2024-04-22T05:50:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f6a56f91640 1 -- 172.21.15.40:0/3292179992 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6a34066760 msgr2=0x7f6a34068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f6a56f91640 1 --2- 172.21.15.40:0/3292179992 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6a34066760 0x7f6a34068c20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a4802c1e0 tx=0x7f6a5012c040 comp rx=0 tx=0).stop 2024-04-22T05:50:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f6a56f91640 1 -- 172.21.15.40:0/3292179992 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a48037500 msgr2=0x7f6a48160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f6a56f91640 1 --2- 172.21.15.40:0/3292179992 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a48037500 0x7f6a48160670 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000d9c0 tx=0x7f6a4000ddc0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f6a56f91640 1 -- 172.21.15.40:0/3292179992 shutdown_connections 2024-04-22T05:50:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f6a56f91640 1 --2- 172.21.15.40:0/3292179992 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6a34066760 0x7f6a34068c20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f6a56f91640 1 --2- 172.21.15.40:0/3292179992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a480c2b80 0x7f6a480c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f6a56f91640 1 --2- 172.21.15.40:0/3292179992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a480c7dc0 0x7f6a480c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f6a56f91640 1 --2- 172.21.15.40:0/3292179992 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a48037500 0x7f6a48160670 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f6a56f91640 1 -- 172.21.15.40:0/3292179992 >> 172.21.15.40:0/3292179992 conn(0x7f6a4803a850 msgr2=0x7f6a48038a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f6a56f91640 1 -- 172.21.15.40:0/3292179992 shutdown_connections 2024-04-22T05:50:44.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.514+0000 7f6a56f91640 1 -- 172.21.15.40:0/3292179992 wait complete. 2024-04-22T05:50:44.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.530+0000 7f7d88fe1640 1 -- 172.21.15.40:0/3624805635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7d8413fd60 con 0x7f7d8413e1f0 2024-04-22T05:50:44.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.530+0000 7f7d6bfff640 1 -- 172.21.15.40:0/3624805635 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d8413fd60 con 0x7f7d8413e1f0 2024-04-22T05:50:44.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542146 got 60129542146 for osd.2 2024-04-22T05:50:44.534 INFO:teuthology.orchestra.run.smithi040.stdout:60129542148 2024-04-22T05:50:44.534 DEBUG:teuthology.parallel:result is None 2024-04-22T05:50:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f7d88fe1640 1 -- 172.21.15.40:0/3624805635 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f7d70066690 msgr2=0x7f7d70068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f7d88fe1640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f7d70066690 0x7f7d70068b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d740af000 tx=0x7f7d74008040 comp rx=0 tx=0).stop 2024-04-22T05:50:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f7d88fe1640 1 -- 172.21.15.40:0/3624805635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8413e1f0 msgr2=0x7f7d84141a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f7d88fe1640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8413e1f0 0x7f7d84141a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c001570 tx=0x7f7d7c00cd20 comp rx=0 tx=0).stop 2024-04-22T05:50:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f7d88fe1640 1 -- 172.21.15.40:0/3624805635 shutdown_connections 2024-04-22T05:50:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f7d88fe1640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f7d70066690 0x7f7d70068b50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f7d88fe1640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8413e1f0 0x7f7d84141a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f7d88fe1640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d840b4210 0x7f7d841414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f7d88fe1640 1 --2- 172.21.15.40:0/3624805635 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f7d840ad1b0 0x7f7d8413eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f7d88fe1640 1 -- 172.21.15.40:0/3624805635 >> 172.21.15.40:0/3624805635 conn(0x7f7d8406b730 msgr2=0x7f7d840b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f7d88fe1640 1 -- 172.21.15.40:0/3624805635 shutdown_connections 2024-04-22T05:50:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f7d88fe1640 1 -- 172.21.15.40:0/3624805635 wait complete. 2024-04-22T05:50:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/382785431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6eb40d2cd0 con 0x7f6eb40c81b0 2024-04-22T05:50:44.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.534+0000 7f6eb27fc640 1 -- 172.21.15.40:0/382785431 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6eac0510e0 con 0x7f6eb40c81b0 2024-04-22T05:50:44.538 INFO:teuthology.orchestra.run.smithi040.stdout:60129542147 2024-04-22T05:50:44.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.538+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/382785431 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6ea0066690 msgr2=0x7f6ea0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.538+0000 7f6ec2a0f640 1 --2- 172.21.15.40:0/382785431 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6ea0066690 0x7f6ea0068b50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc063ce0 tx=0x7f6ebc06c040 comp rx=0 tx=0).stop 2024-04-22T05:50:44.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.538+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/382785431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb40c81b0 msgr2=0x7f6eb40c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.538+0000 7f6ec2a0f640 1 --2- 172.21.15.40:0/382785431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb40c81b0 0x7f6eb40c8630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6eb402cdd0 tx=0x7f6eac00b870 comp rx=0 tx=0).stop 2024-04-22T05:50:44.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.538+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/382785431 shutdown_connections 2024-04-22T05:50:44.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.538+0000 7f6ec2a0f640 1 --2- 172.21.15.40:0/382785431 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6ea0066690 0x7f6ea0068b50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.538+0000 7f6ec2a0f640 1 --2- 172.21.15.40:0/382785431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb40c81b0 0x7f6eb40c8630 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.538+0000 7f6ec2a0f640 1 --2- 172.21.15.40:0/382785431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6eb402c9c0 0x7f6eb40c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.538+0000 7f6ec2a0f640 1 --2- 172.21.15.40:0/382785431 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6eb40072c0 0x7f6eb40c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.538+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/382785431 >> 172.21.15.40:0/382785431 conn(0x7f6eb4006290 msgr2=0x7f6eb40c2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.538+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/382785431 shutdown_connections 2024-04-22T05:50:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.538+0000 7f6ec2a0f640 1 -- 172.21.15.40:0/382785431 wait complete. 2024-04-22T05:50:44.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.1 2024-04-22T05:50:44.543 DEBUG:teuthology.parallel:result is None 2024-04-22T05:50:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.550+0000 7f3628959640 1 -- 172.21.15.40:0/2043012521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f362406f280 con 0x7f362406faa0 2024-04-22T05:50:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.550+0000 7f361affd640 1 -- 172.21.15.40:0/2043012521 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f362406f280 con 0x7f362406faa0 2024-04-22T05:50:44.554 INFO:teuthology.orchestra.run.smithi040.stdout:55834574851 2024-04-22T05:50:44.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.6 2024-04-22T05:50:44.555 DEBUG:teuthology.parallel:result is None 2024-04-22T05:50:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.554+0000 7f3628959640 1 -- 172.21.15.40:0/2043012521 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f3604066760 msgr2=0x7f3604068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.554+0000 7f3628959640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f3604066760 0x7f3604068c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f360c01c710 tx=0x7f360c0a7000 comp rx=0 tx=0).stop 2024-04-22T05:50:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.554+0000 7f3628959640 1 -- 172.21.15.40:0/2043012521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f362406faa0 msgr2=0x7f36241e0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.554+0000 7f3628959640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f362406faa0 0x7f36241e0850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c730 tx=0x7f361c00cbf0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.554+0000 7f3628959640 1 -- 172.21.15.40:0/2043012521 shutdown_connections 2024-04-22T05:50:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.554+0000 7f3628959640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f3604066760 0x7f3604068c20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.554+0000 7f3628959640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f362406faa0 0x7f36241e0850 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.554+0000 7f3628959640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f3624144430 0x7f362406f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.554+0000 7f3628959640 1 --2- 172.21.15.40:0/2043012521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240adf60 0x7f3624134cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.554+0000 7f3628959640 1 -- 172.21.15.40:0/2043012521 >> 172.21.15.40:0/2043012521 conn(0x7f36240b28a0 msgr2=0x7f36240b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.554+0000 7f3628959640 1 -- 172.21.15.40:0/2043012521 shutdown_connections 2024-04-22T05:50:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.554+0000 7f3628959640 1 -- 172.21.15.40:0/2043012521 wait complete. 2024-04-22T05:50:44.557 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.5 2024-04-22T05:50:44.557 DEBUG:teuthology.parallel:result is None 2024-04-22T05:50:44.570 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2024-04-22T05:50:44.571 DEBUG:teuthology.parallel:result is None 2024-04-22T05:50:44.571 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-22T05:50:44.571 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T05:50:44.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff121520640 1 Processor -- start 2024-04-22T05:50:44.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff121520640 1 -- start start 2024-04-22T05:50:44.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff121520640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff11c0aeca0 0x7ff11c06f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff121520640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff11c13adb0 0x7ff11c147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff121520640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11c0af080 con 0x7ff11c06fda0 2024-04-22T05:50:44.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff121520640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11c072350 con 0x7ff11c13adb0 2024-04-22T05:50:44.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff121520640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c071370 con 0x7ff11c0aeca0 2024-04-22T05:50:44.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff11a7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff11c06fda0 0x7ff11c13a6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56118/0 (socket says 172.21.15.40:56118) 2024-04-22T05:50:44.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff11a7fc640 1 -- 172.21.15.40:0/204975517 learned_addr learned my addr 172.21.15.40:0/204975517 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:50:44.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff11affd640 1 --2- 172.21.15.40:0/204975517 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff11c0aeca0 0x7ff11c06f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff11b7fe640 1 --2- 172.21.15.40:0/204975517 >> v2:172.21.15.62:3300/0 conn(0x7ff11c13adb0 0x7ff11c147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff119ffb640 1 -- 172.21.15.40:0/204975517 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514292677 0 0) 0x7ff11c0af080 con 0x7ff11c06fda0 2024-04-22T05:50:44.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff119ffb640 1 -- 172.21.15.40:0/204975517 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0fc003450 con 0x7ff11c06fda0 2024-04-22T05:50:44.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff11affd640 1 -- 172.21.15.40:0/204975517 >> v2:172.21.15.62:3300/0 conn(0x7ff11c13adb0 msgr2=0x7ff11c147ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:44.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff11affd640 1 --2- 172.21.15.40:0/204975517 >> v2:172.21.15.62:3300/0 conn(0x7ff11c13adb0 0x7ff11c147ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff11affd640 1 -- 172.21.15.40:0/204975517 >> v1:172.21.15.40:6789/0 conn(0x7ff11c06fda0 legacy=0x7ff11c13a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.714+0000 7ff11affd640 1 -- 172.21.15.40:0/204975517 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c070200 con 0x7ff11c0aeca0 2024-04-22T05:50:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.718+0000 7ff11affd640 1 --2- 172.21.15.40:0/204975517 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff11c0aeca0 0x7ff11c06f860 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff114009d90 tx=0x7ff11401c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9d13db5891326e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.718+0000 7ff119ffb640 1 -- 172.21.15.40:0/204975517 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1140a5960 con 0x7ff11c0aeca0 2024-04-22T05:50:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.718+0000 7ff119ffb640 1 -- 172.21.15.40:0/204975517 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1140a5ac0 con 0x7ff11c0aeca0 2024-04-22T05:50:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.718+0000 7ff119ffb640 1 -- 172.21.15.40:0/204975517 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1140a5e30 con 0x7ff11c0aeca0 2024-04-22T05:50:44.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 -- 172.21.15.40:0/204975517 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff11c0aeca0 msgr2=0x7ff11c06f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 --2- 172.21.15.40:0/204975517 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff11c0aeca0 0x7ff11c06f860 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff114009d90 tx=0x7ff11401c920 comp rx=0 tx=0).stop 2024-04-22T05:50:44.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 -- 172.21.15.40:0/204975517 shutdown_connections 2024-04-22T05:50:44.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 --2- 172.21.15.40:0/204975517 >> v2:172.21.15.62:3300/0 conn(0x7ff11c13adb0 0x7ff11c147ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 --2- 172.21.15.40:0/204975517 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff11c0aeca0 0x7ff11c06f860 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 -- 172.21.15.40:0/204975517 >> 172.21.15.40:0/204975517 conn(0x7ff11c0afb30 msgr2=0x7ff11c0aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 -- 172.21.15.40:0/204975517 shutdown_connections 2024-04-22T05:50:44.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 -- 172.21.15.40:0/204975517 wait complete. 2024-04-22T05:50:44.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 Processor -- start 2024-04-22T05:50:44.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 -- start start 2024-04-22T05:50:44.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff11c06fda0 0x7ff11c136e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff11c0aeca0 0x7ff11c1d7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff11c13adb0 0x7ff11c1d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0b2430 con 0x7ff11c13adb0 2024-04-22T05:50:44.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0b1fe0 con 0x7ff11c0aeca0 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0b21a0 con 0x7ff11c06fda0 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff11affd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff11c06fda0 0x7ff11c136e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff11affd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff11c06fda0 0x7ff11c136e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35338/0 (socket says 172.21.15.40:35338) 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff11affd640 1 -- 172.21.15.40:0/640084445 learned_addr learned my addr 172.21.15.40:0/640084445 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff11b7fe640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff11c13adb0 0x7ff11c1d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff11a7fc640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff11c0aeca0 0x7ff11c1d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff11b7fe640 1 -- 172.21.15.40:0/640084445 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff11c06fda0 msgr2=0x7ff11c136e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff11b7fe640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff11c06fda0 0x7ff11c136e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff11b7fe640 1 -- 172.21.15.40:0/640084445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff11c0aeca0 msgr2=0x7ff11c1d7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff11b7fe640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff11c0aeca0 0x7ff11c1d7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff11b7fe640 1 -- 172.21.15.40:0/640084445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff114009a70 con 0x7ff11c13adb0 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff11affd640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff11c06fda0 0x7ff11c136e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff11a7fc640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff11c0aeca0 0x7ff11c1d7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:50:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff11b7fe640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff11c13adb0 0x7ff11c1d7da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00e7c0 tx=0x7ff10c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff1137fe640 1 -- 172.21.15.40:0/640084445 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c00c730 con 0x7ff11c13adb0 2024-04-22T05:50:44.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff1137fe640 1 -- 172.21.15.40:0/640084445 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c00c8b0 con 0x7ff11c13adb0 2024-04-22T05:50:44.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff1137fe640 1 -- 172.21.15.40:0/640084445 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c00cc60 con 0x7ff11c13adb0 2024-04-22T05:50:44.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 -- 172.21.15.40:0/640084445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11c0ad7a0 con 0x7ff11c13adb0 2024-04-22T05:50:44.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.722+0000 7ff121520640 1 -- 172.21.15.40:0/640084445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11c1d82e0 con 0x7ff11c13adb0 2024-04-22T05:50:44.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.726+0000 7ff121520640 1 -- 172.21.15.40:0/640084445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11c0b25e0 con 0x7ff11c13adb0 2024-04-22T05:50:44.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.726+0000 7ff1137fe640 1 -- 172.21.15.40:0/640084445 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff10c013470 con 0x7ff11c13adb0 2024-04-22T05:50:44.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.726+0000 7ff1137fe640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff1000666c0 0x7ff100068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:44.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.726+0000 7ff1137fe640 1 -- 172.21.15.40:0/640084445 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff10c083f70 con 0x7ff11c13adb0 2024-04-22T05:50:44.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.726+0000 7ff11affd640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff1000666c0 0x7ff100068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:44.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.726+0000 7ff11affd640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff1000666c0 0x7ff100068b80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff11401c960 tx=0x7ff1140a7070 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:44.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.730+0000 7ff1137fe640 1 -- 172.21.15.40:0/640084445 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7ff11c0b25e0 con 0x7ff11c13adb0 2024-04-22T05:50:44.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.878+0000 7ff121520640 1 -- 172.21.15.40:0/640084445 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff11c0ad620 con 0x7ff1000666c0 2024-04-22T05:50:44.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.882+0000 7ff1137fe640 1 -- 172.21.15.40:0/640084445 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+45560 (secure 0 0 0) 0x7ff1140bd070 con 0x7ff1000666c0 2024-04-22T05:50:44.887 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:50:44.888 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-22T05:50:44.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.886+0000 7ff121520640 1 -- 172.21.15.40:0/640084445 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff1000666c0 msgr2=0x7ff100068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.886+0000 7ff121520640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff1000666c0 0x7ff100068b80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff11401c960 tx=0x7ff1140a7070 comp rx=0 tx=0).stop 2024-04-22T05:50:44.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.886+0000 7ff121520640 1 -- 172.21.15.40:0/640084445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff11c13adb0 msgr2=0x7ff11c1d7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:44.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.886+0000 7ff121520640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff11c13adb0 0x7ff11c1d7da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00e7c0 tx=0x7ff10c00ec80 comp rx=0 tx=0).stop 2024-04-22T05:50:44.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.886+0000 7ff121520640 1 -- 172.21.15.40:0/640084445 shutdown_connections 2024-04-22T05:50:44.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.886+0000 7ff121520640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff1000666c0 0x7ff100068b80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.886+0000 7ff121520640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff11c0aeca0 0x7ff11c1d7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.886+0000 7ff121520640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff11c13adb0 0x7ff11c1d7da0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.886+0000 7ff121520640 1 --2- 172.21.15.40:0/640084445 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff11c06fda0 0x7ff11c136e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:44.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.886+0000 7ff121520640 1 -- 172.21.15.40:0/640084445 >> 172.21.15.40:0/640084445 conn(0x7ff11c0afb30 msgr2=0x7ff11c06eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:44.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.886+0000 7ff121520640 1 -- 172.21.15.40:0/640084445 shutdown_connections 2024-04-22T05:50:44.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:44.886+0000 7ff121520640 1 -- 172.21.15.40:0/640084445 wait complete. 2024-04-22T05:50:44.903 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-04-22T05:50:42.936778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":221676,"kb_used_data":1380,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754753044,"statfs":{"total":773094113280,"available":772867117056,"internally_reserved":0,"allocated":1413120,"data_stored":1464932,"data_compressed":11158,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":12744,"internal_metadata":225431096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T05:50:41.055062+0000","last_change":"2024-04-22T05:50:41.050585+0000","last_active":"2024-04-22T05:50:41.055062+0000","last_peered":"2024-04-22T05:50:41.055062+0000","last_clean":"2024-04-22T05:50:41.055062+0000","last_became_active":"2024-04-22T05:50:41.049696+0000","last_became_peered":"2024-04-22T05:50:41.049696+0000","last_unstale":"2024-04-22T05:50:41.055062+0000","last_undegraded":"2024-04-22T05:50:41.055062+0000","last_fullsized":"2024-04-22T05:50:41.055062+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:00:56.754144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T05:50:41.055654+0000","last_change":"2024-04-22T05:50:41.048959+0000","last_active":"2024-04-22T05:50:41.055654+0000","last_peered":"2024-04-22T05:50:41.055654+0000","last_clean":"2024-04-22T05:50:41.055654+0000","last_became_active":"2024-04-22T05:50:41.048628+0000","last_became_peered":"2024-04-22T05:50:41.048628+0000","last_unstale":"2024-04-22T05:50:41.055654+0000","last_undegraded":"2024-04-22T05:50:41.055654+0000","last_fullsized":"2024-04-22T05:50:41.055654+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:46:57.799120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-22T05:50:42.052581+0000","last_change":"2024-04-22T05:50:41.341624+0000","last_active":"2024-04-22T05:50:42.052581+0000","last_peered":"2024-04-22T05:50:42.052581+0000","last_clean":"2024-04-22T05:50:42.052581+0000","last_became_active":"2024-04-22T05:50:41.340643+0000","last_became_peered":"2024-04-22T05:50:41.340643+0000","last_unstale":"2024-04-22T05:50:42.052581+0000","last_undegraded":"2024-04-22T05:50:42.052581+0000","last_fullsized":"2024-04-22T05:50:42.052581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:47:12.897727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T05:50:41.346273+0000","last_change":"2024-04-22T05:50:41.341548+0000","last_active":"2024-04-22T05:50:41.346273+0000","last_peered":"2024-04-22T05:50:41.346273+0000","last_clean":"2024-04-22T05:50:41.346273+0000","last_became_active":"2024-04-22T05:50:41.341254+0000","last_became_peered":"2024-04-22T05:50:41.341254+0000","last_unstale":"2024-04-22T05:50:41.346273+0000","last_undegraded":"2024-04-22T05:50:41.346273+0000","last_fullsized":"2024-04-22T05:50:41.346273+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:34:01.082016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T05:50:41.054828+0000","last_change":"2024-04-22T05:50:41.051239+0000","last_active":"2024-04-22T05:50:41.054828+0000","last_peered":"2024-04-22T05:50:41.054828+0000","last_clean":"2024-04-22T05:50:41.054828+0000","last_became_active":"2024-04-22T05:50:41.050711+0000","last_became_peered":"2024-04-22T05:50:41.050711+0000","last_unstale":"2024-04-22T05:50:41.054828+0000","last_undegraded":"2024-04-22T05:50:41.054828+0000","last_fullsized":"2024-04-22T05:50:41.054828+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:07:59.435066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T05:50:41.055831+0000","last_change":"2024-04-22T05:50:41.051710+0000","last_active":"2024-04-22T05:50:41.055831+0000","last_peered":"2024-04-22T05:50:41.055831+0000","last_clean":"2024-04-22T05:50:41.055831+0000","last_became_active":"2024-04-22T05:50:41.051259+0000","last_became_peered":"2024-04-22T05:50:41.051259+0000","last_unstale":"2024-04-22T05:50:41.055831+0000","last_undegraded":"2024-04-22T05:50:41.055831+0000","last_fullsized":"2024-04-22T05:50:41.055831+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:41:46.406807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-22T05:50:42.052498+0000","last_change":"2024-04-22T05:50:41.045192+0000","last_active":"2024-04-22T05:50:42.052498+0000","last_peered":"2024-04-22T05:50:42.052498+0000","last_clean":"2024-04-22T05:50:42.052498+0000","last_became_active":"2024-04-22T05:50:41.044865+0000","last_became_peered":"2024-04-22T05:50:41.044865+0000","last_unstale":"2024-04-22T05:50:42.052498+0000","last_undegraded":"2024-04-22T05:50:42.052498+0000","last_fullsized":"2024-04-22T05:50:42.052498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:37:43.523824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T05:50:41.057747+0000","last_change":"2024-04-22T05:50:41.052651+0000","last_active":"2024-04-22T05:50:41.057747+0000","last_peered":"2024-04-22T05:50:41.057747+0000","last_clean":"2024-04-22T05:50:41.057747+0000","last_became_active":"2024-04-22T05:50:41.052349+0000","last_became_peered":"2024-04-22T05:50:41.052349+0000","last_unstale":"2024-04-22T05:50:41.057747+0000","last_undegraded":"2024-04-22T05:50:41.057747+0000","last_fullsized":"2024-04-22T05:50:41.057747+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:46:51.494717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-22T05:50:42.052674+0000","last_change":"2024-04-22T05:50:40.047233+0000","last_active":"2024-04-22T05:50:42.052674+0000","last_peered":"2024-04-22T05:50:42.052674+0000","last_clean":"2024-04-22T05:50:42.052674+0000","last_became_active":"2024-04-22T05:50:40.046329+0000","last_became_peered":"2024-04-22T05:50:40.046329+0000","last_unstale":"2024-04-22T05:50:42.052674+0000","last_undegraded":"2024-04-22T05:50:42.052674+0000","last_fullsized":"2024-04-22T05:50:42.052674+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:39.011105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:39.011105+0000","last_clean_scrub_stamp":"2024-04-22T05:50:39.011105+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:16:02.587654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11158,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":14,"seq":60129542146,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":51005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":51005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.724}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7969999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.752}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8240000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":163840,"data_stored":51005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":51005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6140000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.536}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2810000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6030000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1539999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":212992,"data_stored":578969,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.76}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":217088,"data_stored":579933,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":163840,"data_stored":51005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574850,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":51005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T05:50:44.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T05:50:45.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e905f640 1 Processor -- start 2024-04-22T05:50:45.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e905f640 1 -- start start 2024-04-22T05:50:45.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e905f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb6e413f8b0 0x7fb6e40b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e905f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb6e40afd30 0x7fb6e406ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e905f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e4139140 con 0x7fb6e40af950 2024-04-22T05:50:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e905f640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e4071ca0 con 0x7fb6e413f8b0 2024-04-22T05:50:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e905f640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e40b0140 con 0x7fb6e40afd30 2024-04-22T05:50:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e359e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb6e40afd30 0x7fb6e406ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e2d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb6e40af950 0x7fb6e4138a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56128/0 (socket says 172.21.15.40:56128) 2024-04-22T05:50:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e2d9d640 1 -- 172.21.15.40:0/3598250400 learned_addr learned my addr 172.21.15.40:0/3598250400 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:50:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e259c640 1 --2- 172.21.15.40:0/3598250400 >> v2:172.21.15.62:3300/0 conn(0x7fb6e413f8b0 0x7fb6e40b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e359e640 1 -- 172.21.15.40:0/3598250400 >> v2:172.21.15.62:3300/0 conn(0x7fb6e413f8b0 msgr2=0x7fb6e40b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e359e640 1 --2- 172.21.15.40:0/3598250400 >> v2:172.21.15.62:3300/0 conn(0x7fb6e413f8b0 0x7fb6e40b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e359e640 1 -- 172.21.15.40:0/3598250400 >> v1:172.21.15.40:6789/0 conn(0x7fb6e40af950 legacy=0x7fb6e4138a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e1d9b640 1 -- 172.21.15.40:0/3598250400 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638972061 0 0) 0x7fb6e4139140 con 0x7fb6e40af950 2024-04-22T05:50:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.022+0000 7fb6e359e640 1 -- 172.21.15.40:0/3598250400 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e413f390 con 0x7fb6e40afd30 2024-04-22T05:50:45.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e359e640 1 --2- 172.21.15.40:0/3598250400 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb6e40afd30 0x7fb6e406ff70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40097c0 tx=0x7fb6d401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff533472e1e7da47 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:45.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e1d9b640 1 -- 172.21.15.40:0/3598250400 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40ab600 con 0x7fb6e40afd30 2024-04-22T05:50:45.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e1d9b640 1 -- 172.21.15.40:0/3598250400 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d40a7050 con 0x7fb6e40afd30 2024-04-22T05:50:45.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e1d9b640 1 -- 172.21.15.40:0/3598250400 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40ab980 con 0x7fb6e40afd30 2024-04-22T05:50:45.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 -- 172.21.15.40:0/3598250400 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb6e40afd30 msgr2=0x7fb6e406ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 --2- 172.21.15.40:0/3598250400 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb6e40afd30 0x7fb6e406ff70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40097c0 tx=0x7fb6d401c3f0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 -- 172.21.15.40:0/3598250400 shutdown_connections 2024-04-22T05:50:45.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 --2- 172.21.15.40:0/3598250400 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb6e40afd30 0x7fb6e406ff70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 --2- 172.21.15.40:0/3598250400 >> v2:172.21.15.62:3300/0 conn(0x7fb6e413f8b0 0x7fb6e40b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 -- 172.21.15.40:0/3598250400 >> 172.21.15.40:0/3598250400 conn(0x7fb6e40b4af0 msgr2=0x7fb6e40b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:45.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 -- 172.21.15.40:0/3598250400 shutdown_connections 2024-04-22T05:50:45.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 -- 172.21.15.40:0/3598250400 wait complete. 2024-04-22T05:50:45.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 Processor -- start 2024-04-22T05:50:45.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 -- start start 2024-04-22T05:50:45.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e40af950 0x7fb6e406c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb6e40afd30 0x7fb6e4147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6e413f8b0 0x7fb6e4148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e41477a0 con 0x7fb6e413f8b0 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e4149170 con 0x7fb6e40af950 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e41479f0 con 0x7fb6e40afd30 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6e413f8b0 0x7fb6e4148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6e413f8b0 0x7fb6e4148450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59800/0 (socket says 172.21.15.40:59800) 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e359e640 1 -- 172.21.15.40:0/1735136507 learned_addr learned my addr 172.21.15.40:0/1735136507 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e259c640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb6e40afd30 0x7fb6e4147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e2d9d640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e40af950 0x7fb6e406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e359e640 1 -- 172.21.15.40:0/1735136507 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb6e40afd30 msgr2=0x7fb6e4147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e359e640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb6e40afd30 0x7fb6e4147f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e359e640 1 -- 172.21.15.40:0/1735136507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e40af950 msgr2=0x7fb6e406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e359e640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e40af950 0x7fb6e406c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e359e640 1 -- 172.21.15.40:0/1735136507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d4009420 con 0x7fb6e413f8b0 2024-04-22T05:50:45.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e2d9d640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e40af950 0x7fb6e406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T05:50:45.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e359e640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6e413f8b0 0x7fb6e4148450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6d401a020 tx=0x7fb6d4019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:45.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6db7fe640 1 -- 172.21.15.40:0/1735136507 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40b4840 con 0x7fb6e413f8b0 2024-04-22T05:50:45.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6db7fe640 1 -- 172.21.15.40:0/1735136507 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d40b49a0 con 0x7fb6e413f8b0 2024-04-22T05:50:45.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6e905f640 1 -- 172.21.15.40:0/1735136507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e406cea0 con 0x7fb6e413f8b0 2024-04-22T05:50:45.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.026+0000 7fb6db7fe640 1 -- 172.21.15.40:0/1735136507 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40b4d30 con 0x7fb6e413f8b0 2024-04-22T05:50:45.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.030+0000 7fb6e905f640 1 -- 172.21.15.40:0/1735136507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e4148990 con 0x7fb6e413f8b0 2024-04-22T05:50:45.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.030+0000 7fb6e905f640 1 -- 172.21.15.40:0/1735136507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e406a600 con 0x7fb6e413f8b0 2024-04-22T05:50:45.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.030+0000 7fb6db7fe640 1 -- 172.21.15.40:0/1735136507 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb6d40b2040 con 0x7fb6e413f8b0 2024-04-22T05:50:45.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.030+0000 7fb6db7fe640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fb6c4066690 0x7fb6c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.030+0000 7fb6db7fe640 1 -- 172.21.15.40:0/1735136507 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb6d411e340 con 0x7fb6e413f8b0 2024-04-22T05:50:45.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.030+0000 7fb6e2d9d640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fb6c4066690 0x7fb6c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.030+0000 7fb6e2d9d640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fb6c4066690 0x7fb6c4068b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc005fd0 tx=0x7fb6dc005b80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.034+0000 7fb6db7fe640 1 -- 172.21.15.40:0/1735136507 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7fb6d40eb890 con 0x7fb6e413f8b0 2024-04-22T05:50:45.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.186+0000 7fb6e905f640 1 -- 172.21.15.40:0/1735136507 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6e4066d70 con 0x7fb6c4066690 2024-04-22T05:50:45.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.190+0000 7fb6db7fe640 1 -- 172.21.15.40:0/1735136507 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47062 (secure 0 0 0) 0x7fb6e4066d70 con 0x7fb6c4066690 2024-04-22T05:50:45.195 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:50:45.195 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-22T05:50:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.194+0000 7fb6e905f640 1 -- 172.21.15.40:0/1735136507 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fb6c4066690 msgr2=0x7fb6c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.194+0000 7fb6e905f640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fb6c4066690 0x7fb6c4068b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc005fd0 tx=0x7fb6dc005b80 comp rx=0 tx=0).stop 2024-04-22T05:50:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.194+0000 7fb6e905f640 1 -- 172.21.15.40:0/1735136507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6e413f8b0 msgr2=0x7fb6e4148450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.194+0000 7fb6e905f640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6e413f8b0 0x7fb6e4148450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6d401a020 tx=0x7fb6d4019190 comp rx=0 tx=0).stop 2024-04-22T05:50:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.194+0000 7fb6e905f640 1 -- 172.21.15.40:0/1735136507 shutdown_connections 2024-04-22T05:50:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.194+0000 7fb6e905f640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fb6c4066690 0x7fb6c4068b50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.194+0000 7fb6e905f640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6e413f8b0 0x7fb6e4148450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.194+0000 7fb6e905f640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fb6e40afd30 0x7fb6e4147f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.194+0000 7fb6e905f640 1 --2- 172.21.15.40:0/1735136507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e40af950 0x7fb6e406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.194+0000 7fb6e905f640 1 -- 172.21.15.40:0/1735136507 >> 172.21.15.40:0/1735136507 conn(0x7fb6e40b4af0 msgr2=0x7fb6e4065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:45.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.194+0000 7fb6e905f640 1 -- 172.21.15.40:0/1735136507 shutdown_connections 2024-04-22T05:50:45.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.194+0000 7fb6e905f640 1 -- 172.21.15.40:0/1735136507 wait complete. 2024-04-22T05:50:45.212 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-04-22T05:50:44.937645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":221736,"kb_used_data":1440,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754752984,"statfs":{"total":773094113280,"available":772867055616,"internally_reserved":0,"allocated":1474560,"data_stored":1493672,"data_compressed":11158,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":12744,"internal_metadata":225431096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000867"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T05:50:42.341729+0000","last_change":"2024-04-22T05:50:41.050585+0000","last_active":"2024-04-22T05:50:42.341729+0000","last_peered":"2024-04-22T05:50:42.341729+0000","last_clean":"2024-04-22T05:50:42.341729+0000","last_became_active":"2024-04-22T05:50:41.049696+0000","last_became_peered":"2024-04-22T05:50:41.049696+0000","last_unstale":"2024-04-22T05:50:42.341729+0000","last_undegraded":"2024-04-22T05:50:42.341729+0000","last_fullsized":"2024-04-22T05:50:42.341729+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:00:56.754144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T05:50:41.055654+0000","last_change":"2024-04-22T05:50:41.048959+0000","last_active":"2024-04-22T05:50:41.055654+0000","last_peered":"2024-04-22T05:50:41.055654+0000","last_clean":"2024-04-22T05:50:41.055654+0000","last_became_active":"2024-04-22T05:50:41.048628+0000","last_became_peered":"2024-04-22T05:50:41.048628+0000","last_unstale":"2024-04-22T05:50:41.055654+0000","last_undegraded":"2024-04-22T05:50:41.055654+0000","last_fullsized":"2024-04-22T05:50:41.055654+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:46:57.799120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-22T05:50:42.052581+0000","last_change":"2024-04-22T05:50:41.341624+0000","last_active":"2024-04-22T05:50:42.052581+0000","last_peered":"2024-04-22T05:50:42.052581+0000","last_clean":"2024-04-22T05:50:42.052581+0000","last_became_active":"2024-04-22T05:50:41.340643+0000","last_became_peered":"2024-04-22T05:50:41.340643+0000","last_unstale":"2024-04-22T05:50:42.052581+0000","last_undegraded":"2024-04-22T05:50:42.052581+0000","last_fullsized":"2024-04-22T05:50:42.052581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:47:12.897727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T05:50:41.346273+0000","last_change":"2024-04-22T05:50:41.341548+0000","last_active":"2024-04-22T05:50:41.346273+0000","last_peered":"2024-04-22T05:50:41.346273+0000","last_clean":"2024-04-22T05:50:41.346273+0000","last_became_active":"2024-04-22T05:50:41.341254+0000","last_became_peered":"2024-04-22T05:50:41.341254+0000","last_unstale":"2024-04-22T05:50:41.346273+0000","last_undegraded":"2024-04-22T05:50:41.346273+0000","last_fullsized":"2024-04-22T05:50:41.346273+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:34:01.082016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T05:50:42.342442+0000","last_change":"2024-04-22T05:50:41.051239+0000","last_active":"2024-04-22T05:50:42.342442+0000","last_peered":"2024-04-22T05:50:42.342442+0000","last_clean":"2024-04-22T05:50:42.342442+0000","last_became_active":"2024-04-22T05:50:41.050711+0000","last_became_peered":"2024-04-22T05:50:41.050711+0000","last_unstale":"2024-04-22T05:50:42.342442+0000","last_undegraded":"2024-04-22T05:50:42.342442+0000","last_fullsized":"2024-04-22T05:50:42.342442+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:07:59.435066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T05:50:41.055831+0000","last_change":"2024-04-22T05:50:41.051710+0000","last_active":"2024-04-22T05:50:41.055831+0000","last_peered":"2024-04-22T05:50:41.055831+0000","last_clean":"2024-04-22T05:50:41.055831+0000","last_became_active":"2024-04-22T05:50:41.051259+0000","last_became_peered":"2024-04-22T05:50:41.051259+0000","last_unstale":"2024-04-22T05:50:41.055831+0000","last_undegraded":"2024-04-22T05:50:41.055831+0000","last_fullsized":"2024-04-22T05:50:41.055831+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:41:46.406807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-22T05:50:42.052498+0000","last_change":"2024-04-22T05:50:41.045192+0000","last_active":"2024-04-22T05:50:42.052498+0000","last_peered":"2024-04-22T05:50:42.052498+0000","last_clean":"2024-04-22T05:50:42.052498+0000","last_became_active":"2024-04-22T05:50:41.044865+0000","last_became_peered":"2024-04-22T05:50:41.044865+0000","last_unstale":"2024-04-22T05:50:42.052498+0000","last_undegraded":"2024-04-22T05:50:42.052498+0000","last_fullsized":"2024-04-22T05:50:42.052498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:37:43.523824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T05:50:42.342485+0000","last_change":"2024-04-22T05:50:41.052651+0000","last_active":"2024-04-22T05:50:42.342485+0000","last_peered":"2024-04-22T05:50:42.342485+0000","last_clean":"2024-04-22T05:50:42.342485+0000","last_became_active":"2024-04-22T05:50:41.052349+0000","last_became_peered":"2024-04-22T05:50:41.052349+0000","last_unstale":"2024-04-22T05:50:42.342485+0000","last_undegraded":"2024-04-22T05:50:42.342485+0000","last_fullsized":"2024-04-22T05:50:42.342485+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:46:51.494717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-22T05:50:42.052674+0000","last_change":"2024-04-22T05:50:40.047233+0000","last_active":"2024-04-22T05:50:42.052674+0000","last_peered":"2024-04-22T05:50:42.052674+0000","last_clean":"2024-04-22T05:50:42.052674+0000","last_became_active":"2024-04-22T05:50:40.046329+0000","last_became_peered":"2024-04-22T05:50:40.046329+0000","last_unstale":"2024-04-22T05:50:42.052674+0000","last_undegraded":"2024-04-22T05:50:42.052674+0000","last_fullsized":"2024-04-22T05:50:42.052674+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:39.011105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:39.011105+0000","last_clean_scrub_stamp":"2024-04-22T05:50:39.011105+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:16:02.587654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11158,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":14,"seq":60129542146,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":51005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":176128,"data_stored":56753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.595}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.536}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":56753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":56753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":584717,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.76}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":229376,"data_stored":585681,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":163840,"data_stored":51005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.595}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":51005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T05:50:45.213 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-22T05:50:45.214 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-22T05:50:45.214 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-22T05:50:45.214 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T05:50:45.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.322+0000 7f908e615640 1 Processor -- start 2024-04-22T05:50:45.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.322+0000 7f908e615640 1 -- start start 2024-04-22T05:50:45.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f908e615640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f90880ad500 0x7f9088137650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f908e615640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90880ad8e0 con 0x7f9088137b90 2024-04-22T05:50:45.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f908e615640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9088070dd0 con 0x7f9088141910 2024-04-22T05:50:45.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f908e615640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908806f5a0 con 0x7f90880ad500 2024-04-22T05:50:45.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f90877fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9088137b90 0x7f9088141200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56140/0 (socket says 172.21.15.40:56140) 2024-04-22T05:50:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f90877fe640 1 -- 172.21.15.40:0/1355500262 learned_addr learned my addr 172.21.15.40:0/1355500262 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:50:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9087fff640 1 --2- 172.21.15.40:0/1355500262 >> v2:172.21.15.62:3300/0 conn(0x7f90880ad500 0x7f9088137650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276527556 0 0) 0x7f90880ad8e0 con 0x7f9088137b90 2024-04-22T05:50:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9070003530 con 0x7f9088137b90 2024-04-22T05:50:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1823422877 0 0) 0x7f9070003530 con 0x7f9088137b90 2024-04-22T05:50:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 >> v2:172.21.15.62:3300/0 conn(0x7f90880ad500 msgr2=0x7f9088137650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 --2- 172.21.15.40:0/1355500262 >> v2:172.21.15.62:3300/0 conn(0x7f90880ad500 0x7f9088137650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 >> v1:172.21.15.62:6789/0 conn(0x7f9088141910 legacy=0x7f9088148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9088071040 con 0x7f9088137b90 2024-04-22T05:50:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268782488 0 0) 0x7f9078001680 con 0x7f9088137b90 2024-04-22T05:50:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 >> v1:172.21.15.40:6789/0 conn(0x7f9088137b90 legacy=0x7f9088141200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 --2- 172.21.15.40:0/1355500262 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f90700039f0 0x7f9070003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 --2- 172.21.15.40:0/1355500262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90700047d0 0x7f9070004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 --2- 172.21.15.40:0/1355500262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9070005190 0x7f907000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908806f5a0 con 0x7f9070005190 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907000c0c0 con 0x7f90700047d0 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907000c280 con 0x7f90700039f0 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9087fff640 1 --2- 172.21.15.40:0/1355500262 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f90700039f0 0x7f9070003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f908cbb2640 1 --2- 172.21.15.40:0/1355500262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9070005190 0x7f907000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f90877fe640 1 --2- 172.21.15.40:0/1355500262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90700047d0 0x7f9070004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9087fff640 1 -- 172.21.15.40:0/1355500262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90700047d0 msgr2=0x7f9070004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9087fff640 1 --2- 172.21.15.40:0/1355500262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90700047d0 0x7f9070004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9087fff640 1 -- 172.21.15.40:0/1355500262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9070005190 msgr2=0x7f907000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9087fff640 1 --2- 172.21.15.40:0/1355500262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9070005190 0x7f907000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9087fff640 1 -- 172.21.15.40:0/1355500262 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9070003530 con 0x7f90700039f0 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f908cbb2640 1 --2- 172.21.15.40:0/1355500262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9070005190 0x7f907000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:50:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f908cbb2640 1 -- 172.21.15.40:0/1355500262 reap_dead start 2024-04-22T05:50:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9087fff640 1 --2- 172.21.15.40:0/1355500262 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f90700039f0 0x7f9070003e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f907c009ae0 tx=0x7f907c01ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30969b7d1d1a9e45 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0a5640 con 0x7f90700039f0 2024-04-22T05:50:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c0a57a0 con 0x7f90700039f0 2024-04-22T05:50:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.326+0000 7f9086ffd640 1 -- 172.21.15.40:0/1355500262 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0a5b50 con 0x7f90700039f0 2024-04-22T05:50:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 -- 172.21.15.40:0/1355500262 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f90700039f0 msgr2=0x7f9070003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 --2- 172.21.15.40:0/1355500262 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f90700039f0 0x7f9070003e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f907c009ae0 tx=0x7f907c01ca50 comp rx=0 tx=0).stop 2024-04-22T05:50:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 -- 172.21.15.40:0/1355500262 shutdown_connections 2024-04-22T05:50:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 --2- 172.21.15.40:0/1355500262 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f90700039f0 0x7f9070003e10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 -- 172.21.15.40:0/1355500262 >> 172.21.15.40:0/1355500262 conn(0x7f90880b28a0 msgr2=0x7f90880b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 -- 172.21.15.40:0/1355500262 shutdown_connections 2024-04-22T05:50:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 -- 172.21.15.40:0/1355500262 wait complete. 2024-04-22T05:50:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 Processor -- start 2024-04-22T05:50:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 -- start start 2024-04-22T05:50:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f90880abac0 0x7f9088140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90881d6560 0x7f90881c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90881d9a60 0x7f90881d9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90880abea0 con 0x7f90881d9a60 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90881c7d70 con 0x7f90881d6560 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90881d69c0 con 0x7f90880abac0 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f9087fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f90880abac0 0x7f9088140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f9087fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f90880abac0 0x7f9088140cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35382/0 (socket says 172.21.15.40:35382) 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f9087fff640 1 -- 172.21.15.40:0/3933496634 learned_addr learned my addr 172.21.15.40:0/3933496634 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908cbb2640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90881d9a60 0x7f90881d9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f90877fe640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90881d6560 0x7f90881c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f9087fff640 1 -- 172.21.15.40:0/3933496634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90881d6560 msgr2=0x7f90881c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f9087fff640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90881d6560 0x7f90881c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f9087fff640 1 -- 172.21.15.40:0/3933496634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90881d9a60 msgr2=0x7f90881d9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f9087fff640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90881d9a60 0x7f90881d9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f9087fff640 1 -- 172.21.15.40:0/3933496634 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90881c8010 con 0x7f90880abac0 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f9087fff640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f90880abac0 0x7f9088140cc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f907c01c880 tx=0x7f907c0197f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:45.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f9084ff9640 1 -- 172.21.15.40:0/3933496634 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0a5640 con 0x7f90880abac0 2024-04-22T05:50:45.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f9084ff9640 1 -- 172.21.15.40:0/3933496634 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c0b66a0 con 0x7f90880abac0 2024-04-22T05:50:45.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 -- 172.21.15.40:0/3933496634 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90881c8a70 con 0x7f90880abac0 2024-04-22T05:50:45.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f9084ff9640 1 -- 172.21.15.40:0/3933496634 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0b6a50 con 0x7f90880abac0 2024-04-22T05:50:45.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 -- 172.21.15.40:0/3933496634 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90881cbb90 con 0x7f90880abac0 2024-04-22T05:50:45.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.330+0000 7f908e615640 1 -- 172.21.15.40:0/3933496634 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90881c81a0 con 0x7f90880abac0 2024-04-22T05:50:45.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.334+0000 7f9084ff9640 1 -- 172.21.15.40:0/3933496634 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f907c0b4070 con 0x7f90880abac0 2024-04-22T05:50:45.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.334+0000 7f9084ff9640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9074066690 0x7f9074068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.334+0000 7f9084ff9640 1 -- 172.21.15.40:0/3933496634 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f907c11dd80 con 0x7f90880abac0 2024-04-22T05:50:45.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.334+0000 7f90877fe640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9074066690 0x7f9074068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.334+0000 7f90877fe640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9074066690 0x7f9074068b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9078000fe0 tx=0x7f9078009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:45.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.338+0000 7f9084ff9640 1 -- 172.21.15.40:0/3933496634 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f907c0eb5c0 con 0x7f90880abac0 2024-04-22T05:50:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.522+0000 7f908e615640 1 -- 172.21.15.40:0/3933496634 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f90880aee70 con 0x7f90880abac0 2024-04-22T05:50:45.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.522+0000 7f9084ff9640 1 -- 172.21.15.40:0/3933496634 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f907c0eebe0 con 0x7f90880abac0 2024-04-22T05:50:45.525 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:50:45.525 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-22T05:50:45.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.526+0000 7f908e615640 1 -- 172.21.15.40:0/3933496634 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9074066690 msgr2=0x7f9074068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.526+0000 7f908e615640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9074066690 0x7f9074068b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9078000fe0 tx=0x7f9078009040 comp rx=0 tx=0).stop 2024-04-22T05:50:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.526+0000 7f908e615640 1 -- 172.21.15.40:0/3933496634 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f90880abac0 msgr2=0x7f9088140cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.526+0000 7f908e615640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f90880abac0 0x7f9088140cc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f907c01c880 tx=0x7f907c0197f0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.526+0000 7f908e615640 1 -- 172.21.15.40:0/3933496634 shutdown_connections 2024-04-22T05:50:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.526+0000 7f908e615640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9074066690 0x7f9074068b50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.526+0000 7f908e615640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90881d6560 0x7f90881c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.526+0000 7f908e615640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90881d9a60 0x7f90881d9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.526+0000 7f908e615640 1 --2- 172.21.15.40:0/3933496634 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f90880abac0 0x7f9088140cc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.526+0000 7f908e615640 1 -- 172.21.15.40:0/3933496634 >> 172.21.15.40:0/3933496634 conn(0x7f90880b28a0 msgr2=0x7f90880b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.526+0000 7f908e615640 1 -- 172.21.15.40:0/3933496634 shutdown_connections 2024-04-22T05:50:45.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.526+0000 7f908e615640 1 -- 172.21.15.40:0/3933496634 wait complete. 2024-04-22T05:50:45.543 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-22T05:50:45.543 INFO:teuthology.run_tasks:Running task exec... 2024-04-22T05:50:45.555 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T05:50:45.555 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi040.front.sepia.ceph.com 2024-04-22T05:50:45.555 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool create cephsqlite' 2024-04-22T05:50:45.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.670+0000 7f178536c640 1 Processor -- start 2024-04-22T05:50:45.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.670+0000 7f178536c640 1 -- start start 2024-04-22T05:50:45.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f178536c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f178013e3c0 0x7f178013e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f178536c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17800b56e0 con 0x7f17800b5300 2024-04-22T05:50:45.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f178536c640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178006b470 con 0x7f17800aeb90 2024-04-22T05:50:45.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f178536c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178006b6c0 con 0x7f178013e3c0 2024-04-22T05:50:45.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f177effd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f17800b5300 0x7f17800af700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56154/0 (socket says 172.21.15.40:56154) 2024-04-22T05:50:45.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f177effd640 1 -- 172.21.15.40:0/3152190967 learned_addr learned my addr 172.21.15.40:0/3152190967 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:50:45.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f177f7fe640 1 --2- 172.21.15.40:0/3152190967 >> v2:172.21.15.40:3300/0 conn(0x7f178013e3c0 0x7f178013e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f177dffb640 1 -- 172.21.15.40:0/3152190967 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997959968 0 0) 0x7f17800b56e0 con 0x7f17800b5300 2024-04-22T05:50:45.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f177dffb640 1 -- 172.21.15.40:0/3152190967 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1768003530 con 0x7f17800b5300 2024-04-22T05:50:45.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f177f7fe640 1 -- 172.21.15.40:0/3152190967 >> v1:172.21.15.62:6789/0 conn(0x7f17800aeb90 legacy=0x7f178006bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f177f7fe640 1 -- 172.21.15.40:0/3152190967 >> v1:172.21.15.40:6789/0 conn(0x7f17800b5300 legacy=0x7f17800af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f177f7fe640 1 -- 172.21.15.40:0/3152190967 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17800afee0 con 0x7f178013e3c0 2024-04-22T05:50:45.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f177f7fe640 1 --2- 172.21.15.40:0/3152190967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f178013e3c0 0x7f178013e7d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1778009d90 tx=0x7f177801c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a3b8f8d4fcf3e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:45.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f177dffb640 1 -- 172.21.15.40:0/3152190967 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17780a5800 con 0x7f178013e3c0 2024-04-22T05:50:45.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f177dffb640 1 -- 172.21.15.40:0/3152190967 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17780a5960 con 0x7f178013e3c0 2024-04-22T05:50:45.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f177dffb640 1 -- 172.21.15.40:0/3152190967 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17780a5cf0 con 0x7f178013e3c0 2024-04-22T05:50:45.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f178536c640 1 -- 172.21.15.40:0/3152190967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f178013e3c0 msgr2=0x7f178013e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f178536c640 1 --2- 172.21.15.40:0/3152190967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f178013e3c0 0x7f178013e7d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1778009d90 tx=0x7f177801c830 comp rx=0 tx=0).stop 2024-04-22T05:50:45.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f178536c640 1 -- 172.21.15.40:0/3152190967 shutdown_connections 2024-04-22T05:50:45.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f178536c640 1 --2- 172.21.15.40:0/3152190967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f178013e3c0 0x7f178013e7d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f178536c640 1 -- 172.21.15.40:0/3152190967 >> 172.21.15.40:0/3152190967 conn(0x7f1780073ce0 msgr2=0x7f17800b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:45.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f178536c640 1 -- 172.21.15.40:0/3152190967 shutdown_connections 2024-04-22T05:50:45.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.674+0000 7f178536c640 1 -- 172.21.15.40:0/3152190967 wait complete. 2024-04-22T05:50:45.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f178536c640 1 Processor -- start 2024-04-22T05:50:45.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f178536c640 1 -- start start 2024-04-22T05:50:45.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f178536c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17800aeb90 0x7f1780140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f178536c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17800b5300 0x7f17801d6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f178536c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f178013e3c0 0x7f178013abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f178536c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17801d89e0 con 0x7f17800aeb90 2024-04-22T05:50:45.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f178536c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17801d8bd0 con 0x7f17800b5300 2024-04-22T05:50:45.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f178536c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17800abe60 con 0x7f178013e3c0 2024-04-22T05:50:45.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f177effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17800aeb90 0x7f1780140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f177f7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f178013e3c0 0x7f178013abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f177f7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f178013e3c0 0x7f178013abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35386/0 (socket says 172.21.15.40:35386) 2024-04-22T05:50:45.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f177f7fe640 1 -- 172.21.15.40:0/1631645464 learned_addr learned my addr 172.21.15.40:0/1631645464 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:45.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f177e7fc640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17800b5300 0x7f17801d6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f177f7fe640 1 -- 172.21.15.40:0/1631645464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17800b5300 msgr2=0x7f17801d6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f177f7fe640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17800b5300 0x7f17801d6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f177f7fe640 1 -- 172.21.15.40:0/1631645464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17800aeb90 msgr2=0x7f1780140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:45.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f177f7fe640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17800aeb90 0x7f1780140b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:45.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f177f7fe640 1 -- 172.21.15.40:0/1631645464 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1778009a70 con 0x7f178013e3c0 2024-04-22T05:50:45.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f177e7fc640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17800b5300 0x7f17801d6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:50:45.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f177f7fe640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f178013e3c0 0x7f178013abc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f17780b0040 tx=0x7f17780195d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:45.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f17677fe640 1 -- 172.21.15.40:0/1631645464 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17780adb80 con 0x7f178013e3c0 2024-04-22T05:50:45.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f17677fe640 1 -- 172.21.15.40:0/1631645464 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17780adce0 con 0x7f178013e3c0 2024-04-22T05:50:45.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f178536c640 1 -- 172.21.15.40:0/1631645464 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1780141470 con 0x7f178013e3c0 2024-04-22T05:50:45.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f17677fe640 1 -- 172.21.15.40:0/1631645464 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17780b9650 con 0x7f178013e3c0 2024-04-22T05:50:45.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f178536c640 1 -- 172.21.15.40:0/1631645464 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17801d9340 con 0x7f178013e3c0 2024-04-22T05:50:45.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f178536c640 1 -- 172.21.15.40:0/1631645464 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17801d9060 con 0x7f178013e3c0 2024-04-22T05:50:45.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f17677fe640 1 -- 172.21.15.40:0/1631645464 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f17780b97b0 con 0x7f178013e3c0 2024-04-22T05:50:45.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f17677fe640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f176c0666c0 0x7f176c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:45.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.678+0000 7f17677fe640 1 -- 172.21.15.40:0/1631645464 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f177811f4c0 con 0x7f178013e3c0 2024-04-22T05:50:45.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.682+0000 7f177effd640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f176c0666c0 0x7f176c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:45.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.682+0000 7f177effd640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f176c0666c0 0x7f176c068b80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f17700011f0 tx=0x7f1770007d40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:45.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.682+0000 7f17677fe640 1 -- 172.21.15.40:0/1631645464 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f17801d9060 con 0x7f178013e3c0 2024-04-22T05:50:45.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:45.842+0000 7f178536c640 1 -- 172.21.15.40:0/1631645464 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephsqlite"} v 0) v1 -- 0x7f17801d9060 con 0x7f178013e3c0 2024-04-22T05:50:46.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.086+0000 7f17677fe640 1 -- 172.21.15.40:0/1631645464 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephsqlite"}]=0 pool 'cephsqlite' created v19) v1 ==== 110+0+0 (secure 0 0 0) 0x7f17801d9060 con 0x7f178013e3c0 2024-04-22T05:50:46.087 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephsqlite' created 2024-04-22T05:50:46.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.090+0000 7f178536c640 1 -- 172.21.15.40:0/1631645464 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f176c0666c0 msgr2=0x7f176c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:46.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.090+0000 7f178536c640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f176c0666c0 0x7f176c068b80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f17700011f0 tx=0x7f1770007d40 comp rx=0 tx=0).stop 2024-04-22T05:50:46.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.090+0000 7f178536c640 1 -- 172.21.15.40:0/1631645464 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f178013e3c0 msgr2=0x7f178013abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:46.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.090+0000 7f178536c640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f178013e3c0 0x7f178013abc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f17780b0040 tx=0x7f17780195d0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.090+0000 7f178536c640 1 -- 172.21.15.40:0/1631645464 shutdown_connections 2024-04-22T05:50:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.090+0000 7f178536c640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f176c0666c0 0x7f176c068b80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.090+0000 7f178536c640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17800b5300 0x7f17801d6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.090+0000 7f178536c640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f178013e3c0 0x7f178013abc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.090+0000 7f178536c640 1 --2- 172.21.15.40:0/1631645464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17800aeb90 0x7f1780140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.090+0000 7f178536c640 1 -- 172.21.15.40:0/1631645464 >> 172.21.15.40:0/1631645464 conn(0x7f1780073ce0 msgr2=0x7f17801381d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.098+0000 7f178536c640 1 -- 172.21.15.40:0/1631645464 shutdown_connections 2024-04-22T05:50:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.098+0000 7f178536c640 1 -- 172.21.15.40:0/1631645464 wait complete. 2024-04-22T05:50:46.120 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph auth get-or-create client.libcephsqlite mon '"'"'profile simple-rados-client-with-blocklist'"'"' osd '"'"'allow rwx pool=cephsqlite'"'"' >> /etc/ceph/ceph.keyring' 2024-04-22T05:50:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.242+0000 7f8a6e2c8640 1 Processor -- start 2024-04-22T05:50:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.242+0000 7f8a6e2c8640 1 -- start start 2024-04-22T05:50:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a6e2c8640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8a68069e50 0x7f8a680ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a6e2c8640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8a6806a230 0x7f8a680af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a6e2c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a680abc80 0x7f8a68141000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a6e2c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a680acd60 con 0x7f8a680abc80 2024-04-22T05:50:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a6e2c8640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a680ad9b0 con 0x7f8a6806a230 2024-04-22T05:50:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a6e2c8640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a680aded0 con 0x7f8a68069e50 2024-04-22T05:50:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a67fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8a68069e50 0x7f8a680ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a6c865640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a680abc80 0x7f8a68141000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a67fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8a68069e50 0x7f8a680ad3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35398/0 (socket says 172.21.15.40:35398) 2024-04-22T05:50:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a67fff640 1 -- 172.21.15.40:0/1961417107 learned_addr learned my addr 172.21.15.40:0/1961417107 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a677fe640 1 --2- 172.21.15.40:0/1961417107 >> v2:172.21.15.62:3300/0 conn(0x7f8a6806a230 0x7f8a680af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a67fff640 1 -- 172.21.15.40:0/1961417107 >> v2:172.21.15.62:3300/0 conn(0x7f8a6806a230 msgr2=0x7f8a680af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a67fff640 1 --2- 172.21.15.40:0/1961417107 >> v2:172.21.15.62:3300/0 conn(0x7f8a6806a230 0x7f8a680af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a67fff640 1 -- 172.21.15.40:0/1961417107 >> v2:172.21.15.40:3300/0 conn(0x7f8a680abc80 msgr2=0x7f8a68141000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:50:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a67fff640 1 --2- 172.21.15.40:0/1961417107 >> v2:172.21.15.40:3300/0 conn(0x7f8a680abc80 0x7f8a68141000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a67fff640 1 -- 172.21.15.40:0/1961417107 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a680ae040 con 0x7f8a68069e50 2024-04-22T05:50:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a6c865640 1 --2- 172.21.15.40:0/1961417107 >> v2:172.21.15.40:3300/0 conn(0x7f8a680abc80 0x7f8a68141000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:50:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a67fff640 1 --2- 172.21.15.40:0/1961417107 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8a68069e50 0x7f8a680ad3d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009870 tx=0x7f8a5801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8b944c393c70e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a66ffd640 1 -- 172.21.15.40:0/1961417107 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a580ab600 con 0x7f8a68069e50 2024-04-22T05:50:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a66ffd640 1 -- 172.21.15.40:0/1961417107 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a580a7050 con 0x7f8a68069e50 2024-04-22T05:50:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.246+0000 7f8a66ffd640 1 -- 172.21.15.40:0/1961417107 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a580ab980 con 0x7f8a68069e50 2024-04-22T05:50:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/1961417107 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8a68069e50 msgr2=0x7f8a680ad3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 --2- 172.21.15.40:0/1961417107 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8a68069e50 0x7f8a680ad3d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009870 tx=0x7f8a5801c470 comp rx=0 tx=0).stop 2024-04-22T05:50:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/1961417107 shutdown_connections 2024-04-22T05:50:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 --2- 172.21.15.40:0/1961417107 >> v2:172.21.15.62:3300/0 conn(0x7f8a6806a230 0x7f8a680af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 --2- 172.21.15.40:0/1961417107 >> v2:172.21.15.40:3300/0 conn(0x7f8a680abc80 0x7f8a68141000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 --2- 172.21.15.40:0/1961417107 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8a68069e50 0x7f8a680ad3d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/1961417107 >> 172.21.15.40:0/1961417107 conn(0x7f8a680b0da0 msgr2=0x7f8a680b3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/1961417107 shutdown_connections 2024-04-22T05:50:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/1961417107 wait complete. 2024-04-22T05:50:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 Processor -- start 2024-04-22T05:50:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 -- start start 2024-04-22T05:50:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8a68069e50 0x7f8a6813fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a680abc80 0x7f8a680b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680b2910 0x7f8a681dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a681dc6f0 con 0x7f8a680b2910 2024-04-22T05:50:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6813ee90 con 0x7f8a680abc80 2024-04-22T05:50:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a680b2d90 con 0x7f8a68069e50 2024-04-22T05:50:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a67fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8a68069e50 0x7f8a6813fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6c865640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680b2910 0x7f8a681dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a67fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8a68069e50 0x7f8a6813fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:35402/0 (socket says 172.21.15.40:35402) 2024-04-22T05:50:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a67fff640 1 -- 172.21.15.40:0/390169316 learned_addr learned my addr 172.21.15.40:0/390169316 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:50:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a677fe640 1 --2- 172.21.15.40:0/390169316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a680abc80 0x7f8a680b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6c865640 1 -- 172.21.15.40:0/390169316 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8a68069e50 msgr2=0x7f8a6813fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6c865640 1 --2- 172.21.15.40:0/390169316 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8a68069e50 0x7f8a6813fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6c865640 1 -- 172.21.15.40:0/390169316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a680abc80 msgr2=0x7f8a680b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6c865640 1 --2- 172.21.15.40:0/390169316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a680abc80 0x7f8a680b1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6c865640 1 -- 172.21.15.40:0/390169316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a60009a70 con 0x7f8a680b2910 2024-04-22T05:50:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6c865640 1 --2- 172.21.15.40:0/390169316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680b2910 0x7f8a681dcc10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000efc0 tx=0x7f8a6000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a64ff9640 1 -- 172.21.15.40:0/390169316 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000cd30 con 0x7f8a680b2910 2024-04-22T05:50:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a64ff9640 1 -- 172.21.15.40:0/390169316 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6000f040 con 0x7f8a680b2910 2024-04-22T05:50:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/390169316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a580094d0 con 0x7f8a680b2910 2024-04-22T05:50:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a64ff9640 1 -- 172.21.15.40:0/390169316 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60003be0 con 0x7f8a680b2910 2024-04-22T05:50:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.250+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/390169316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a681dd320 con 0x7f8a680b2910 2024-04-22T05:50:46.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.254+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/390169316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a680673b0 con 0x7f8a680b2910 2024-04-22T05:50:46.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.254+0000 7f8a64ff9640 1 -- 172.21.15.40:0/390169316 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8a60014480 con 0x7f8a680b2910 2024-04-22T05:50:46.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.254+0000 7f8a64ff9640 1 --2- 172.21.15.40:0/390169316 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8a54066690 0x7f8a54068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:50:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.254+0000 7f8a64ff9640 1 -- 172.21.15.40:0/390169316 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5559+0+0 (secure 0 0 0) 0x7f8a60084a30 con 0x7f8a680b2910 2024-04-22T05:50:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.254+0000 7f8a67fff640 1 --2- 172.21.15.40:0/390169316 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8a54066690 0x7f8a54068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:50:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.254+0000 7f8a67fff640 1 --2- 172.21.15.40:0/390169316 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8a54066690 0x7f8a54068b50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8a5801c4b0 tx=0x7f8a58008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:50:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.258+0000 7f8a64ff9640 1 -- 172.21.15.40:0/390169316 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f8a60051ec0 con 0x7f8a680b2910 2024-04-22T05:50:46.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.474+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/390169316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.libcephsqlite", "caps": ["mon", "profile simple-rados-client-with-blocklist", "osd", "allow rwx pool=cephsqlite"]} v 0) v1 -- 0x7f8a6806ee10 con 0x7f8a680b2910 2024-04-22T05:50:46.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.482+0000 7f8a64ff9640 1 -- 172.21.15.40:0/390169316 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.libcephsqlite", "caps": ["mon", "profile simple-rados-client-with-blocklist", "osd", "allow rwx pool=cephsqlite"]}]=0 v9) v1 ==== 199+0+71 (secure 0 0 0) 0x7f8a60018060 con 0x7f8a680b2910 2024-04-22T05:50:46.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.482+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/390169316 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8a54066690 msgr2=0x7f8a54068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:46.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.482+0000 7f8a6e2c8640 1 --2- 172.21.15.40:0/390169316 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8a54066690 0x7f8a54068b50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8a5801c4b0 tx=0x7f8a58008040 comp rx=0 tx=0).stop 2024-04-22T05:50:46.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.482+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/390169316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680b2910 msgr2=0x7f8a681dcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:50:46.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.482+0000 7f8a6e2c8640 1 --2- 172.21.15.40:0/390169316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680b2910 0x7f8a681dcc10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000efc0 tx=0x7f8a6000c490 comp rx=0 tx=0).stop 2024-04-22T05:50:46.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.482+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/390169316 shutdown_connections 2024-04-22T05:50:46.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.482+0000 7f8a6e2c8640 1 --2- 172.21.15.40:0/390169316 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8a54066690 0x7f8a54068b50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.482+0000 7f8a6e2c8640 1 --2- 172.21.15.40:0/390169316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a680abc80 0x7f8a680b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.482+0000 7f8a6e2c8640 1 --2- 172.21.15.40:0/390169316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680b2910 0x7f8a681dcc10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.482+0000 7f8a6e2c8640 1 --2- 172.21.15.40:0/390169316 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8a68069e50 0x7f8a6813fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:50:46.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.482+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/390169316 >> 172.21.15.40:0/390169316 conn(0x7f8a680b0da0 msgr2=0x7f8a68143960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:50:46.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.482+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/390169316 shutdown_connections 2024-04-22T05:50:46.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:50:46.482+0000 7f8a6e2c8640 1 -- 172.21.15.40:0/390169316 wait complete. 2024-04-22T05:50:46.502 INFO:teuthology.run_tasks:Running task exec... 2024-04-22T05:50:46.511 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T05:50:46.511 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi040.front.sepia.ceph.com 2024-04-22T05:50:46.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr' 2024-04-22T05:50:46.550 INFO:teuthology.orchestra.run.smithi040.stdout:[==========] Running 20 tests from 1 test suite. 2024-04-22T05:50:46.551 INFO:teuthology.orchestra.run.smithi040.stdout:[----------] Global test environment set-up. 2024-04-22T05:50:46.551 INFO:teuthology.orchestra.run.smithi040.stdout:[----------] 20 tests from CephSQLiteTest 2024-04-22T05:50:51.558 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.Create 2024-04-22T05:50:51.575 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/5f1ebe14-c3c3-44ad-abf2-ab8d449db91a.db?vfs=ceph 2024-04-22T05:50:51.575 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:50:51.583 INFO:teuthology.orchestra.run.smithi040.stdout:CREATE TABLE foo (a INT); 2024-04-22T05:50:51.824 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.Create (266 ms) 2024-04-22T05:50:51.824 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.InsertBulk4096 2024-04-22T05:50:51.844 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/a2b9c108-d0ec-466e-9a45-f2855e555b6e.db?vfs=ceph 2024-04-22T05:50:51.844 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:50:51.851 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA page_size = 4096;CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOM() FROM c LIMIT 1000000;PRAGMA page_size; 2024-04-22T05:50:54.297 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.InsertBulk4096 (2473 ms) 2024-04-22T05:50:54.297 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.InsertBulk 2024-04-22T05:50:54.308 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/004ca09e-0cc4-460f-b53c-9fff347994a1.db?vfs=ceph 2024-04-22T05:50:54.308 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:50:54.524 INFO:teuthology.orchestra.run.smithi040.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOM() FROM c LIMIT 1000000; 2024-04-22T05:50:55.423 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.InsertBulk (1127 ms) 2024-04-22T05:50:55.424 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.UpdateBulk 2024-04-22T05:50:55.437 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/e53c0266-6a17-4223-a6e6-90b96d39e8a2.db?vfs=ceph 2024-04-22T05:50:55.437 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:50:55.442 INFO:teuthology.orchestra.run.smithi040.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;SELECT SUM(a) FROM foo;UPDATE foo SET a = a+a;SELECT SUM(a) FROM foo; 2024-04-22T05:50:56.793 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.UpdateBulk (1369 ms) 2024-04-22T05:50:56.793 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.InsertRate 2024-04-22T05:50:56.805 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/0a9d3b92-19fb-41b6-8ea6-54308b865825.db?vfs=ceph 2024-04-22T05:50:56.805 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:50:56.813 INFO:teuthology.orchestra.run.smithi040.stdout:CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-04-22T05:51:00.580 INFO:teuthology.orchestra.run.smithi040.stdout:transactions per second: 26.7382 2024-04-22T05:51:00.581 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.InsertRate (3788 ms) 2024-04-22T05:51:00.581 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.DatabaseShrink 2024-04-22T05:51:00.595 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/36603831-8fa1-411d-8bd4-569ceb048cb7.db?vfs=ceph 2024-04-22T05:51:00.595 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:51:00.604 INFO:teuthology.orchestra.run.smithi040.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;DELETE FROM foo WHERE RANDOM()%4 < 3;VACUUM; 2024-04-22T05:51:02.542 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.DatabaseShrink (1962 ms) 2024-04-22T05:51:02.542 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveRate 2024-04-22T05:51:02.560 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/0b83639a-f1b4-4330-85d2-2bcbe2147a7c.db?vfs=ceph 2024-04-22T05:51:02.560 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:51:02.567 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA locking_mode=EXCLUSIVE;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-04-22T05:51:04.245 INFO:teuthology.orchestra.run.smithi040.stdout:transactions per second: 60.387 2024-04-22T05:51:04.249 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.InsertExclusiveRate (1707 ms) 2024-04-22T05:51:04.250 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveWALRate 2024-04-22T05:51:04.265 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/24863029-2281-4ca2-8f86-bf621f48b29d.db?vfs=ceph 2024-04-22T05:51:04.266 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:51:04.275 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA locking_mode=EXCLUSIVE;PRAGMA journal_mode=WAL;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-04-22T05:51:04.915 INFO:teuthology.orchestra.run.smithi040.stdout:transactions per second: 174.827 2024-04-22T05:51:04.949 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.InsertExclusiveWALRate (699 ms) 2024-04-22T05:51:04.949 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.WALTransactionSync 2024-04-22T05:51:04.962 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/0ecbae77-9da3-48c6-bb82-bdcd91c86c6b.db?vfs=ceph 2024-04-22T05:51:04.962 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:51:04.968 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA locking_mode=EXCLUSIVE;PRAGMA journal_mode=WAL;CREATE TABLE foo (a INT);INSERT INTO perf (v) VALUES (ceph_perf());BEGIN TRANSACTION;INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount'; 2024-04-22T05:51:05.065 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.WALTransactionSync (117 ms) 2024-04-22T05:51:05.066 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.PersistTransactionSync 2024-04-22T05:51:05.078 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/f21bc54d-12ef-49ca-94ca-3f5b8a096d14.db?vfs=ceph 2024-04-22T05:51:05.078 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:51:05.084 INFO:teuthology.orchestra.run.smithi040.stdout:BEGIN TRANSACTION;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount'; 2024-04-22T05:51:05.122 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.PersistTransactionSync (57 ms) 2024-04-22T05:51:05.123 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveLock 2024-04-22T05:51:05.136 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/8959eba7-7a74-44b5-8e78-555991c4c26a.db?vfs=ceph 2024-04-22T05:51:05.136 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:51:05.145 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA locking_mode=EXCLUSIVE;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_lock.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_lock.avgcount';SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.lock' AND b.fullkey = '$.libcephsqlite_striper.lock'; 2024-04-22T05:51:05.188 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.InsertExclusiveLock (66 ms) 2024-04-22T05:51:05.188 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.TransactionSizeUpdate 2024-04-22T05:51:05.205 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/e2eeb2d8-4f28-424c-80e4-18c1fc94a3cb.db?vfs=ceph 2024-04-22T05:51:05.205 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:51:05.214 INFO:teuthology.orchestra.run.smithi040.stdout:BEGIN TRANSACTION;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.update_size' AND b.fullkey = '$.libcephsqlite_striper.update_size'; 2024-04-22T05:51:05.258 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.TransactionSizeUpdate (70 ms) 2024-04-22T05:51:05.259 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.AllocatedGrowth 2024-04-22T05:51:05.271 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/675680e1-2ca6-472e-b4a5-51ec2c4df458.db?vfs=ceph 2024-04-22T05:51:05.271 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:51:05.278 INFO:teuthology.orchestra.run.smithi040.stdout:CREATE TABLE foo (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.update_allocated' AND b.fullkey = '$.libcephsqlite_striper.update_allocated'; 2024-04-22T05:51:14.057 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.AllocatedGrowth (8797 ms) 2024-04-22T05:51:14.058 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.DeleteBulk 2024-04-22T05:51:14.065 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/531506fe-0bb3-4aa9-aeb7-6c0993888090.db?vfs=ceph 2024-04-22T05:51:14.065 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:51:14.071 INFO:teuthology.orchestra.run.smithi040.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;DELETE FROM foo WHERE RANDOM()%2 == 0; 2024-04-22T05:51:15.138 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.DeleteBulk (1082 ms) 2024-04-22T05:51:15.138 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.DropMassive 2024-04-22T05:51:15.153 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/60a43f69-cd8a-466d-b365-0acb0de2a4e2.db?vfs=ceph 2024-04-22T05:51:15.153 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:51:15.158 INFO:teuthology.orchestra.run.smithi040.stdout:CREATE TABLE foo (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;DROP TABLE foo;VACUUM;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.shrink' AND b.fullkey = '$.libcephsqlite_striper.shrink';SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.shrink_bytes' AND b.fullkey = '$.libcephsqlite_striper.shrink_bytes'; 2024-04-22T05:51:52.725 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.DropMassive (37587 ms) 2024-04-22T05:51:52.725 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.InsertMassiveVerify 2024-04-22T05:51:52.733 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/bae097d4-8385-42a8-bf17-803b8922b797.db?vfs=ceph 2024-04-22T05:51:52.734 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:51:52.738 INFO:teuthology.orchestra.run.smithi040.stdout:CREATE TABLE foo (a BLOB);CREATE TEMPORARY TABLE bar (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO bar (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;SELECT a FROM bar;INSERT INTO foo (a) SELECT a FROM bar;SELECT a FROM foo; 2024-04-22T05:52:08.044 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.InsertMassiveVerify (15319 ms) 2024-04-22T05:52:08.045 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.PerfValid 2024-04-22T05:52:08.053 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/58bb12ca-9394-4d3b-bf0e-a8a96b665cc5.db?vfs=ceph 2024-04-22T05:52:08.053 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:52:08.060 INFO:teuthology.orchestra.run.smithi040.stdout:SELECT json_valid(ceph_perf()); 2024-04-22T05:52:08.061 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.PerfValid (17 ms) 2024-04-22T05:52:08.061 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.StatusValid 2024-04-22T05:52:08.070 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/b001168b-67a0-45cf-b366-f9b67067fa31.db?vfs=ceph 2024-04-22T05:52:08.070 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:52:08.075 INFO:teuthology.orchestra.run.smithi040.stdout:SELECT json_valid(ceph_status()); 2024-04-22T05:52:08.075 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.StatusValid (15 ms) 2024-04-22T05:52:08.075 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.CurrentTime 2024-04-22T05:52:08.293 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/7acb043b-14d8-41fe-aa04-a8361f435eba.db?vfs=ceph 2024-04-22T05:52:08.293 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:52:08.305 INFO:teuthology.orchestra.run.smithi040.stdout:SELECT strftime('%s', 'now'); 2024-04-22T05:52:08.306 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.CurrentTime (230 ms) 2024-04-22T05:52:08.306 INFO:teuthology.orchestra.run.smithi040.stdout:[ RUN ] CephSQLiteTest.StatusFields 2024-04-22T05:52:08.319 INFO:teuthology.orchestra.run.smithi040.stdout:using database: file:cephsqlite:/985c07fc-2f4d-4bb6-b5e6-fa9a07258d3e.db?vfs=ceph 2024-04-22T05:52:08.319 INFO:teuthology.orchestra.run.smithi040.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-22T05:52:08.328 INFO:teuthology.orchestra.run.smithi040.stdout:SELECT json_extract(ceph_status(), '$.addr');SELECT json_extract(ceph_status(), '$.id'); 2024-04-22T05:52:08.328 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2568233877 2024-04-22T05:52:08.328 INFO:teuthology.orchestra.run.smithi040.stdout:4407 2024-04-22T05:52:08.328 INFO:teuthology.orchestra.run.smithi040.stdout:[ OK ] CephSQLiteTest.StatusFields (23 ms) 2024-04-22T05:52:08.329 INFO:teuthology.orchestra.run.smithi040.stdout:[----------] 20 tests from CephSQLiteTest (76771 ms total) 2024-04-22T05:52:08.329 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:52:08.329 INFO:teuthology.orchestra.run.smithi040.stdout:[----------] Global test environment tear-down 2024-04-22T05:52:08.329 INFO:teuthology.orchestra.run.smithi040.stdout:[==========] 20 tests from 1 test suite ran. (81778 ms total) 2024-04-22T05:52:08.329 INFO:teuthology.orchestra.run.smithi040.stdout:[ PASSED ] 20 tests. 2024-04-22T05:52:08.390 INFO:teuthology.run_tasks:Running task workunit... 2024-04-22T05:52:08.403 INFO:tasks.workunit:Pulling workunits from ref c3a345d31ea77c01771428ab80e1cf32c9b05875 2024-04-22T05:52:08.404 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-22T05:52:08.404 DEBUG:teuthology.orchestra.run.smithi040:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-22T05:52:08.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T05:52:08.440 INFO:teuthology.orchestra.run.smithi040.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-22T05:52:08.440 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-22T05:52:08.496 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-22T05:52:08.496 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-22T05:52:08.543 INFO:tasks.workunit:timeout=3h 2024-04-22T05:52:08.543 INFO:tasks.workunit:cleanup=True 2024-04-22T05:52:08.544 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout c3a345d31ea77c01771428ab80e1cf32c9b05875 2024-04-22T05:52:08.593 INFO:tasks.workunit.client.0.smithi040.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-22T05:53:17.017 INFO:tasks.workunit.client.0.smithi040.stderr:Note: switching to 'c3a345d31ea77c01771428ab80e1cf32c9b05875'. 2024-04-22T05:53:17.018 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-04-22T05:53:17.018 INFO:tasks.workunit.client.0.smithi040.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T05:53:17.018 INFO:tasks.workunit.client.0.smithi040.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T05:53:17.018 INFO:tasks.workunit.client.0.smithi040.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T05:53:17.018 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-04-22T05:53:17.018 INFO:tasks.workunit.client.0.smithi040.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T05:53:17.018 INFO:tasks.workunit.client.0.smithi040.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T05:53:17.018 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-04-22T05:53:17.018 INFO:tasks.workunit.client.0.smithi040.stderr: git switch -c 2024-04-22T05:53:17.018 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-04-22T05:53:17.018 INFO:tasks.workunit.client.0.smithi040.stderr:Or undo this operation with: 2024-04-22T05:53:17.019 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-04-22T05:53:17.019 INFO:tasks.workunit.client.0.smithi040.stderr: git switch - 2024-04-22T05:53:17.019 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-04-22T05:53:17.019 INFO:tasks.workunit.client.0.smithi040.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T05:53:17.019 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-04-22T05:53:17.019 INFO:tasks.workunit.client.0.smithi040.stderr:HEAD is now at c3a345d31ea Merge pull request #57018 from zdover23/wip-doc-2024-04-21-backport-56904-to-reef 2024-04-22T05:53:17.024 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-22T05:53:17.071 INFO:tasks.workunit.client.0.smithi040.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T05:53:17.073 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T05:53:17.074 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T05:53:17.830 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T05:53:17.917 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T05:53:17.979 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T05:53:17.982 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T05:53:17.982 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T05:53:18.033 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T05:53:18.037 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-22T05:53:18.037 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-22T05:53:18.087 INFO:tasks.workunit:Running workunits matching rados/test_libcephsqlite.sh on client.0... 2024-04-22T05:53:18.088 INFO:tasks.workunit:Running workunit rados/test_libcephsqlite.sh... 2024-04-22T05:53:18.089 DEBUG:teuthology.orchestra.run.smithi040:workunit test rados/test_libcephsqlite.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c3a345d31ea77c01771428ab80e1cf32c9b05875 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CEPH_ARGS='--id libcephsqlite --no-log-to-stderr' adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh cephsqlite 2024-04-22T05:53:18.144 INFO:tasks.workunit.client.0.smithi040.stderr:+ pool=cephsqlite 2024-04-22T05:53:18.145 INFO:tasks.workunit.client.0.smithi040.stderr:++ basename /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh 2024-04-22T05:53:18.146 INFO:tasks.workunit.client.0.smithi040.stderr:+ ns=test_libcephsqlite.sh 2024-04-22T05:53:18.146 INFO:tasks.workunit.client.0.smithi040.stderr:+ striper rm baz.db 2024-04-22T05:53:18.146 INFO:tasks.workunit.client.0.smithi040.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-04-22T05:53:18.210 INFO:tasks.workunit.client.0.smithi040.stderr:+ sqlite 2024-04-22T05:53:18.210 INFO:tasks.workunit.client.0.smithi040.stderr:+ background= 2024-04-22T05:53:18.211 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' '' = b ']' 2024-04-22T05:53:18.211 INFO:tasks.workunit.client.0.smithi040.stderr:++ cat 2024-04-22T05:53:18.211 INFO:tasks.workunit.client.0.smithi040.stderr:+ a='create table if not exists foo (a INT); 2024-04-22T05:53:18.211 INFO:tasks.workunit.client.0.smithi040.stderr:insert into foo (a) values (RANDOM()); 2024-04-22T05:53:18.212 INFO:tasks.workunit.client.0.smithi040.stderr:drop table foo;' 2024-04-22T05:53:18.212 INFO:tasks.workunit.client.0.smithi040.stderr:+ printf %s 'create table if not exists foo (a INT); 2024-04-22T05:53:18.212 INFO:tasks.workunit.client.0.smithi040.stderr:insert into foo (a) values (RANDOM()); 2024-04-22T05:53:18.212 INFO:tasks.workunit.client.0.smithi040.stderr:drop table foo;' 2024-04-22T05:53:18.212 INFO:tasks.workunit.client.0.smithi040.stderr:create table if not exists foo (a INT); 2024-04-22T05:53:18.212 INFO:tasks.workunit.client.0.smithi040.stderr:insert into foo (a) values (RANDOM()); 2024-04-22T05:53:18.212 INFO:tasks.workunit.client.0.smithi040.stderr:drop table foo;+ '[' '' '!=' b ']' 2024-04-22T05:53:18.212 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 2024-04-22T05:53:18.212 INFO:tasks.workunit.client.0.smithi040.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-22T05:53:18.559 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-04-22T05:53:18.559 INFO:tasks.workunit.client.0.smithi040.stderr:real 0m0.348s 2024-04-22T05:53:18.559 INFO:tasks.workunit.client.0.smithi040.stderr:user 0m0.068s 2024-04-22T05:53:18.559 INFO:tasks.workunit.client.0.smithi040.stderr:sys 0m0.023s 2024-04-22T05:53:18.559 INFO:tasks.workunit.client.0.smithi040.stderr:+ striper stat baz.db 2024-04-22T05:53:18.559 INFO:tasks.workunit.client.0.smithi040.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper stat baz.db 2024-04-22T05:53:18.604 INFO:tasks.workunit.client.0.smithi040.stdout:cephsqlite/baz.db mtime 2024-04-22T05:53:18.000000+0000, size 8192 2024-04-22T05:53:18.607 INFO:tasks.workunit.client.0.smithi040.stderr:+ striper rm baz.db 2024-04-22T05:53:18.607 INFO:tasks.workunit.client.0.smithi040.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-04-22T05:53:18.662 INFO:tasks.workunit.client.0.smithi040.stderr:+ sqlite 2024-04-22T05:53:18.662 INFO:tasks.workunit.client.0.smithi040.stderr:++ repeat 10 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:18.662 INFO:tasks.workunit.client.0.smithi040.stderr:++ n=10 2024-04-22T05:53:18.662 INFO:tasks.workunit.client.0.smithi040.stderr:++ shift 2024-04-22T05:53:18.662 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i = 0 )) 2024-04-22T05:53:18.662 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 10 )) 2024-04-22T05:53:18.662 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:18.662 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 10 )) 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 10 )) 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 10 )) 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 10 )) 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 10 )) 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:18.663 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:18.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 10 )) 2024-04-22T05:53:18.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:18.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:18.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 10 )) 2024-04-22T05:53:18.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:18.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:18.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 10 )) 2024-04-22T05:53:18.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:18.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:18.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 10 )) 2024-04-22T05:53:18.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:18.664 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:18.665 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 10 )) 2024-04-22T05:53:18.665 INFO:tasks.workunit.client.0.smithi040.stderr:+ background= 2024-04-22T05:53:18.665 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' '' = b ']' 2024-04-22T05:53:18.665 INFO:tasks.workunit.client.0.smithi040.stderr:++ cat 2024-04-22T05:53:18.665 INFO:tasks.workunit.client.0.smithi040.stderr:+ a='CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-04-22T05:53:18.665 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.665 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.665 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.665 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.665 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.665 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.665 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.666 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.666 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.666 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.666 INFO:tasks.workunit.client.0.smithi040.stderr:SELECT LENGTH(text) FROM rand; 2024-04-22T05:53:18.666 INFO:tasks.workunit.client.0.smithi040.stderr:DROP TABLE rand;' 2024-04-22T05:53:18.666 INFO:tasks.workunit.client.0.smithi040.stderr:+ printf %s 'CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-04-22T05:53:18.666 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.666 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.666 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.666 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.666 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.666 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:SELECT LENGTH(text) FROM rand; 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:DROP TABLE rand;' 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.667 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.668 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.668 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:18.668 INFO:tasks.workunit.client.0.smithi040.stderr:SELECT LENGTH(text) FROM rand; 2024-04-22T05:53:18.668 INFO:tasks.workunit.client.0.smithi040.stderr:DROP TABLE rand;+ '[' '' '!=' b ']' 2024-04-22T05:53:18.668 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 2024-04-22T05:53:18.668 INFO:tasks.workunit.client.0.smithi040.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-22T05:53:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.114 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-04-22T05:53:20.114 INFO:tasks.workunit.client.0.smithi040.stderr:real 0m1.446s 2024-04-22T05:53:20.114 INFO:tasks.workunit.client.0.smithi040.stderr:user 0m0.186s 2024-04-22T05:53:20.115 INFO:tasks.workunit.client.0.smithi040.stderr:sys 0m0.081s 2024-04-22T05:53:20.115 INFO:tasks.workunit.client.0.smithi040.stderr:+ sqlite 2024-04-22T05:53:20.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ repeat 100 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ n=100 2024-04-22T05:53:20.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ shift 2024-04-22T05:53:20.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i = 0 )) 2024-04-22T05:53:20.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.115 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.116 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.116 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.116 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.116 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.116 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.116 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.116 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.116 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.117 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.118 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.119 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.120 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.121 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.122 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.123 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.124 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.124 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.124 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.124 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.124 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.124 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.124 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.124 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.124 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.124 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.124 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.124 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.124 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.125 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.126 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.127 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.128 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.129 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.129 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.129 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.129 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.129 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.129 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.129 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.129 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.129 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.129 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.129 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.129 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.130 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.131 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.132 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.132 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.132 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.132 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.132 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.132 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.132 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.132 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.132 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.132 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.133 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.133 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.133 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.133 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.133 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.133 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.133 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.133 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.133 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.133 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.133 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.133 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.133 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.134 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.134 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.134 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.134 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.134 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.134 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.134 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.134 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.134 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.134 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.134 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.134 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.135 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.136 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.136 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.136 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.136 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.136 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.136 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.136 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.136 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.136 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.136 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.136 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.136 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.136 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.137 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.137 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.137 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.137 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.137 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.137 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.137 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.137 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.137 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.137 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.138 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.138 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.138 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.138 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.138 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.138 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.138 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.138 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.138 INFO:tasks.workunit.client.0.smithi040.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-22T05:53:20.138 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( ++i )) 2024-04-22T05:53:20.138 INFO:tasks.workunit.client.0.smithi040.stderr:++ (( i < 100 )) 2024-04-22T05:53:20.138 INFO:tasks.workunit.client.0.smithi040.stderr:+ background= 2024-04-22T05:53:20.138 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' '' = b ']' 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:++ cat 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:+ a='BEGIN TRANSACTION; 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.139 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.140 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.140 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.140 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.140 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.140 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.140 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.140 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.140 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.140 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.140 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.140 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.140 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.140 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.141 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.142 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.143 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.143 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.143 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.143 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.143 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.143 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.143 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.143 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.143 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.143 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.143 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.143 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.143 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.144 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.145 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:COMMIT; 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:SELECT LENGTH(text) FROM rand; 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:DROP TABLE rand;' 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:+ printf %s 'BEGIN TRANSACTION; 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.146 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.147 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.147 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.147 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.147 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.147 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.147 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.147 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.147 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.147 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.147 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.147 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.147 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.147 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.148 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.149 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.150 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.150 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.150 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.150 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.150 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.150 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.150 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.150 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.150 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.151 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.152 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.153 INFO:tasks.workunit.client.0.smithi040.stderr:COMMIT; 2024-04-22T05:53:20.154 INFO:tasks.workunit.client.0.smithi040.stderr:SELECT LENGTH(text) FROM rand; 2024-04-22T05:53:20.154 INFO:tasks.workunit.client.0.smithi040.stderr:DROP TABLE rand;' 2024-04-22T05:53:20.154 INFO:tasks.workunit.client.0.smithi040.stderr:BEGIN TRANSACTION; 2024-04-22T05:53:20.154 INFO:tasks.workunit.client.0.smithi040.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-04-22T05:53:20.154 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.154 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.154 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.154 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.154 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.154 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.154 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.154 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.155 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.156 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.157 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.158 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.159 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.160 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.161 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.161 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.161 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.161 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-22T05:53:20.161 INFO:tasks.workunit.client.0.smithi040.stderr:COMMIT; 2024-04-22T05:53:20.161 INFO:tasks.workunit.client.0.smithi040.stderr:SELECT LENGTH(text) FROM rand; 2024-04-22T05:53:20.161 INFO:tasks.workunit.client.0.smithi040.stderr:DROP TABLE rand;+ '[' '' '!=' b ']' 2024-04-22T05:53:20.161 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 2024-04-22T05:53:20.161 INFO:tasks.workunit.client.0.smithi040.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-22T05:53:20.751 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.751 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.751 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.751 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.751 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.751 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.751 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.751 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.751 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.751 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.753 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.754 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.754 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.754 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.754 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.754 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.754 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.754 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.754 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.754 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.754 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.754 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.754 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.754 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.758 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.758 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.758 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.758 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.758 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.758 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.758 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.758 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:20.758 INFO:tasks.workunit.client.0.smithi040.stdout:4096 2024-04-22T05:53:21.937 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-04-22T05:53:21.937 INFO:tasks.workunit.client.0.smithi040.stderr:real 0m1.831s 2024-04-22T05:53:21.937 INFO:tasks.workunit.client.0.smithi040.stderr:user 0m0.246s 2024-04-22T05:53:21.937 INFO:tasks.workunit.client.0.smithi040.stderr:sys 0m0.125s 2024-04-22T05:53:21.937 INFO:tasks.workunit.client.0.smithi040.stderr:+ striper rm baz.db 2024-04-22T05:53:21.937 INFO:tasks.workunit.client.0.smithi040.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-04-22T05:53:21.997 INFO:tasks.workunit.client.0.smithi040.stderr:+ date 2024-04-22T05:53:21.998 INFO:tasks.workunit.client.0.smithi040.stdout:Mon Apr 22 05:53:22 AM UTC 2024 2024-04-22T05:53:21.998 INFO:tasks.workunit.client.0.smithi040.stderr:+ sqlite b 2024-04-22T05:53:21.999 INFO:tasks.workunit.client.0.smithi040.stderr:+ background=b 2024-04-22T05:53:21.999 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' b = b ']' 2024-04-22T05:53:21.999 INFO:tasks.workunit.client.0.smithi040.stderr:+ shift 2024-04-22T05:53:21.999 INFO:tasks.workunit.client.0.smithi040.stderr:++ cat 2024-04-22T05:53:22.000 INFO:tasks.workunit.client.0.smithi040.stderr:+ a='CREATE TABLE foo (a BLOB); 2024-04-22T05:53:22.000 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO foo VALUES ("start"); 2024-04-22T05:53:22.000 INFO:tasks.workunit.client.0.smithi040.stderr:WITH RECURSIVE c(x) AS 2024-04-22T05:53:22.000 INFO:tasks.workunit.client.0.smithi040.stderr: ( 2024-04-22T05:53:22.000 INFO:tasks.workunit.client.0.smithi040.stderr: VALUES(1) 2024-04-22T05:53:22.000 INFO:tasks.workunit.client.0.smithi040.stderr: UNION ALL 2024-04-22T05:53:22.000 INFO:tasks.workunit.client.0.smithi040.stderr: SELECT x+1 2024-04-22T05:53:22.000 INFO:tasks.workunit.client.0.smithi040.stderr: FROM c 2024-04-22T05:53:22.000 INFO:tasks.workunit.client.0.smithi040.stderr: ) 2024-04-22T05:53:22.000 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO foo (a) 2024-04-22T05:53:22.000 INFO:tasks.workunit.client.0.smithi040.stderr: SELECT RANDOMBLOB(1<<20) 2024-04-22T05:53:22.000 INFO:tasks.workunit.client.0.smithi040.stderr: FROM c 2024-04-22T05:53:22.000 INFO:tasks.workunit.client.0.smithi040.stderr: LIMIT (1<<20);' 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr:+ printf %s 'CREATE TABLE foo (a BLOB); 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO foo VALUES ("start"); 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr:WITH RECURSIVE c(x) AS 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr: ( 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr: VALUES(1) 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr: UNION ALL 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr: SELECT x+1 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr: FROM c 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr: ) 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO foo (a) 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr: SELECT RANDOMBLOB(1<<20) 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr: FROM c 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr: LIMIT (1<<20);' 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr:CREATE TABLE foo (a BLOB); 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO foo VALUES ("start"); 2024-04-22T05:53:22.001 INFO:tasks.workunit.client.0.smithi040.stderr:WITH RECURSIVE c(x) AS 2024-04-22T05:53:22.002 INFO:tasks.workunit.client.0.smithi040.stderr: ( 2024-04-22T05:53:22.002 INFO:tasks.workunit.client.0.smithi040.stderr: VALUES(1) 2024-04-22T05:53:22.002 INFO:tasks.workunit.client.0.smithi040.stderr: UNION ALL 2024-04-22T05:53:22.002 INFO:tasks.workunit.client.0.smithi040.stderr: SELECT x+1 2024-04-22T05:53:22.002 INFO:tasks.workunit.client.0.smithi040.stderr: FROM c 2024-04-22T05:53:22.002 INFO:tasks.workunit.client.0.smithi040.stderr: ) 2024-04-22T05:53:22.002 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO foo (a) 2024-04-22T05:53:22.002 INFO:tasks.workunit.client.0.smithi040.stderr: SELECT RANDOMBLOB(1<<20) 2024-04-22T05:53:22.002 INFO:tasks.workunit.client.0.smithi040.stderr: FROM c 2024-04-22T05:53:22.002 INFO:tasks.workunit.client.0.smithi040.stderr: LIMIT (1<<20);+ '[' b '!=' b ']' 2024-04-22T05:53:22.002 INFO:tasks.workunit.client.0.smithi040.stderr:+ sleep 10 2024-04-22T05:53:22.002 INFO:tasks.workunit.client.0.smithi040.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-22T05:53:32.001 INFO:tasks.workunit.client.0.smithi040.stdout:done 2024-04-22T05:53:32.001 INFO:tasks.workunit.client.0.smithi040.stdout:[1]+ 29904 Running sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" & 2024-04-22T05:53:32.001 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo done 2024-04-22T05:53:32.001 INFO:tasks.workunit.client.0.smithi040.stderr:+ jobs -l 2024-04-22T05:53:32.003 INFO:tasks.workunit.client.0.smithi040.stderr:++ jobs -p 2024-04-22T05:53:32.004 INFO:tasks.workunit.client.0.smithi040.stderr:+ kill -KILL -- 29904 2024-04-22T05:53:32.004 INFO:tasks.workunit.client.0.smithi040.stderr:+ date 2024-04-22T05:53:32.005 INFO:tasks.workunit.client.0.smithi040.stdout:Mon Apr 22 05:53:32 AM UTC 2024 2024-04-22T05:53:32.005 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 2024-04-22T05:53:32.008 INFO:tasks.workunit.client.0.smithi040.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh: line 95: 29904 Killed sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" 2024-04-22T05:53:32.008 INFO:tasks.workunit.client.0.smithi040.stderr:+ date 2024-04-22T05:53:32.010 INFO:tasks.workunit.client.0.smithi040.stdout:Mon Apr 22 05:53:32 AM UTC 2024 2024-04-22T05:53:32.010 INFO:tasks.workunit.client.0.smithi040.stderr:++ sqlite 2024-04-22T05:53:32.011 INFO:tasks.workunit.client.0.smithi040.stderr:++ background= 2024-04-22T05:53:32.011 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' = b ']' 2024-04-22T05:53:32.011 INFO:tasks.workunit.client.0.smithi040.stderr:+++ cat 2024-04-22T05:53:32.012 INFO:tasks.workunit.client.0.smithi040.stderr:++ a='SELECT COUNT(*) FROM foo;' 2024-04-22T05:53:32.012 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %s 'SELECT COUNT(*) FROM foo;' 2024-04-22T05:53:32.012 INFO:tasks.workunit.client.0.smithi040.stderr:SELECT COUNT(*) FROM foo;++ '[' '' '!=' b ']' 2024-04-22T05:53:32.012 INFO:tasks.workunit.client.0.smithi040.stderr:++ wait 2024-04-22T05:53:32.012 INFO:tasks.workunit.client.0.smithi040.stderr:++ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-22T05:54:02.235 INFO:tasks.workunit.client.0.smithi040.stderr:+ n=1 2024-04-22T05:54:02.235 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2024-04-22T05:54:02.235 INFO:tasks.workunit.client.0.smithi040.stderr:+ striper rm baz.db 2024-04-22T05:54:02.235 INFO:tasks.workunit.client.0.smithi040.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-04-22T05:54:02.319 INFO:tasks.workunit.client.0.smithi040.stderr:+ date 2024-04-22T05:54:02.319 INFO:tasks.workunit.client.0.smithi040.stdout:Mon Apr 22 05:54:02 AM UTC 2024 2024-04-22T05:54:02.320 INFO:tasks.workunit.client.0.smithi040.stderr:+ sqlite b 2024-04-22T05:54:02.320 INFO:tasks.workunit.client.0.smithi040.stderr:+ background=b 2024-04-22T05:54:02.320 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' b = b ']' 2024-04-22T05:54:02.320 INFO:tasks.workunit.client.0.smithi040.stderr:+ shift 2024-04-22T05:54:02.320 INFO:tasks.workunit.client.0.smithi040.stderr:++ cat 2024-04-22T05:54:02.321 INFO:tasks.workunit.client.0.smithi040.stderr:+ a='CREATE TABLE foo (a BLOB); 2024-04-22T05:54:02.322 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO foo VALUES ("start"); 2024-04-22T05:54:02.322 INFO:tasks.workunit.client.0.smithi040.stderr:WITH RECURSIVE c(x) AS 2024-04-22T05:54:02.322 INFO:tasks.workunit.client.0.smithi040.stderr: ( 2024-04-22T05:54:02.322 INFO:tasks.workunit.client.0.smithi040.stderr: VALUES(1) 2024-04-22T05:54:02.322 INFO:tasks.workunit.client.0.smithi040.stderr: UNION ALL 2024-04-22T05:54:02.322 INFO:tasks.workunit.client.0.smithi040.stderr: SELECT x+1 2024-04-22T05:54:02.322 INFO:tasks.workunit.client.0.smithi040.stderr: FROM c 2024-04-22T05:54:02.322 INFO:tasks.workunit.client.0.smithi040.stderr: ) 2024-04-22T05:54:02.322 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO foo (a) 2024-04-22T05:54:02.322 INFO:tasks.workunit.client.0.smithi040.stderr: SELECT RANDOMBLOB(1<<20) 2024-04-22T05:54:02.322 INFO:tasks.workunit.client.0.smithi040.stderr: FROM c 2024-04-22T05:54:02.322 INFO:tasks.workunit.client.0.smithi040.stderr: LIMIT (1<<20);' 2024-04-22T05:54:02.322 INFO:tasks.workunit.client.0.smithi040.stderr:+ printf %s 'CREATE TABLE foo (a BLOB); 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO foo VALUES ("start"); 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr:WITH RECURSIVE c(x) AS 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr: ( 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr: VALUES(1) 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr: UNION ALL 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr: SELECT x+1 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr: FROM c 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr: ) 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO foo (a) 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr: SELECT RANDOMBLOB(1<<20) 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr: FROM c 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr: LIMIT (1<<20);' 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr:CREATE TABLE foo (a BLOB); 2024-04-22T05:54:02.323 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO foo VALUES ("start"); 2024-04-22T05:54:02.324 INFO:tasks.workunit.client.0.smithi040.stderr:WITH RECURSIVE c(x) AS 2024-04-22T05:54:02.324 INFO:tasks.workunit.client.0.smithi040.stderr: ( 2024-04-22T05:54:02.324 INFO:tasks.workunit.client.0.smithi040.stderr: VALUES(1) 2024-04-22T05:54:02.324 INFO:tasks.workunit.client.0.smithi040.stderr: UNION ALL 2024-04-22T05:54:02.324 INFO:tasks.workunit.client.0.smithi040.stderr: SELECT x+1 2024-04-22T05:54:02.324 INFO:tasks.workunit.client.0.smithi040.stderr: FROM c 2024-04-22T05:54:02.324 INFO:tasks.workunit.client.0.smithi040.stderr: ) 2024-04-22T05:54:02.324 INFO:tasks.workunit.client.0.smithi040.stderr:INSERT INTO foo (a) 2024-04-22T05:54:02.324 INFO:tasks.workunit.client.0.smithi040.stderr: SELECT RANDOMBLOB(1<<20) 2024-04-22T05:54:02.324 INFO:tasks.workunit.client.0.smithi040.stderr: FROM c 2024-04-22T05:54:02.324 INFO:tasks.workunit.client.0.smithi040.stderr: LIMIT (1<<20);+ '[' b '!=' b ']' 2024-04-22T05:54:02.324 INFO:tasks.workunit.client.0.smithi040.stderr:+ sleep 20 2024-04-22T05:54:02.324 INFO:tasks.workunit.client.0.smithi040.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-22T05:54:22.323 INFO:tasks.workunit.client.0.smithi040.stderr:+ jobs -l 2024-04-22T05:54:22.323 INFO:tasks.workunit.client.0.smithi040.stdout:[1]+ 29972 Running sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" & 2024-04-22T05:54:22.323 INFO:tasks.workunit.client.0.smithi040.stderr:++ jobs -p 2024-04-22T05:54:22.324 INFO:tasks.workunit.client.0.smithi040.stderr:+ kill -STOP -- 29972 2024-04-22T05:54:22.324 INFO:tasks.workunit.client.0.smithi040.stderr:+ sleep 45 2024-04-22T05:55:07.325 INFO:tasks.workunit.client.0.smithi040.stderr:+ date 2024-04-22T05:55:07.326 INFO:tasks.workunit.client.0.smithi040.stdout:Mon Apr 22 05:55:07 AM UTC 2024 2024-04-22T05:55:07.327 INFO:tasks.workunit.client.0.smithi040.stderr:++ jobs -p 2024-04-22T05:55:07.327 INFO:tasks.workunit.client.0.smithi040.stderr:+ kill -CONT -- 29972 2024-04-22T05:55:07.327 INFO:tasks.workunit.client.0.smithi040.stderr:+ sleep 10 2024-04-22T05:55:07.364 INFO:tasks.workunit.client.0.smithi040.stderr:Error: near line 3: stepping, disk I/O error (10) 2024-04-22T05:55:17.329 INFO:tasks.workunit.client.0.smithi040.stderr:+ date 2024-04-22T05:55:17.330 INFO:tasks.workunit.client.0.smithi040.stdout:Mon Apr 22 05:55:17 AM UTC 2024 2024-04-22T05:55:17.330 INFO:tasks.workunit.client.0.smithi040.stderr:+ wait 2024-04-22T05:55:17.330 INFO:tasks.workunit.client.0.smithi040.stderr:+ date 2024-04-22T05:55:17.331 INFO:tasks.workunit.client.0.smithi040.stdout:Mon Apr 22 05:55:17 AM UTC 2024 2024-04-22T05:55:17.332 INFO:tasks.workunit.client.0.smithi040.stderr:++ sqlite 2024-04-22T05:55:17.332 INFO:tasks.workunit.client.0.smithi040.stderr:++ background= 2024-04-22T05:55:17.333 INFO:tasks.workunit.client.0.smithi040.stderr:++ '[' '' = b ']' 2024-04-22T05:55:17.333 INFO:tasks.workunit.client.0.smithi040.stderr:+++ cat 2024-04-22T05:55:17.334 INFO:tasks.workunit.client.0.smithi040.stderr:++ a='SELECT COUNT(*) FROM foo;' 2024-04-22T05:55:17.334 INFO:tasks.workunit.client.0.smithi040.stderr:++ printf %s 'SELECT COUNT(*) FROM foo;' 2024-04-22T05:55:17.334 INFO:tasks.workunit.client.0.smithi040.stderr:SELECT COUNT(*) FROM foo;++ '[' '' '!=' b ']' 2024-04-22T05:55:17.334 INFO:tasks.workunit.client.0.smithi040.stderr:++ wait 2024-04-22T05:55:17.334 INFO:tasks.workunit.client.0.smithi040.stderr:++ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-22T05:55:18.253 INFO:tasks.workunit.client.0.smithi040.stderr:+ n=1 2024-04-22T05:55:18.253 INFO:tasks.workunit.client.0.smithi040.stderr:+ '[' 1 -eq 1 ']' 2024-04-22T05:55:18.253 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T05:55:18.254 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T05:55:18.308 INFO:tasks.workunit:Stopping ['rados/test_libcephsqlite.sh cephsqlite'] on client.0... 2024-04-22T05:55:18.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-22T05:55:18.666 DEBUG:teuthology.parallel:result is None 2024-04-22T05:55:18.667 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T05:55:18.679 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T05:55:18.679 DEBUG:teuthology.orchestra.run.smithi040:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-22T05:55:18.738 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T05:55:18.738 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-22T05:55:18.748 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-22T05:55:18.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T05:55:18.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.906+0000 7fa44bc45640 1 Processor -- start 2024-04-22T05:55:18.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.906+0000 7fa44bc45640 1 -- start start 2024-04-22T05:55:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa44bc45640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4440a79a0 0x7fa4440a7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa44bc45640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44413f8b0 con 0x7fa44413e730 2024-04-22T05:55:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa44bc45640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4440a8bb0 con 0x7fa44406df40 2024-04-22T05:55:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa44bc45640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4440a8e70 con 0x7fa4440a79a0 2024-04-22T05:55:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa4499e1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4440a79a0 0x7fa4440a7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa4499e1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4440a79a0 0x7fa4440a7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47024/0 (socket says 172.21.15.40:47024) 2024-04-22T05:55:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa44a1e2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa44413e730 0x7fa444149b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38548/0 (socket says 172.21.15.40:38548) 2024-04-22T05:55:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa4499e1640 1 -- 172.21.15.40:0/3014482629 learned_addr learned my addr 172.21.15.40:0/3014482629 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa4499e1640 1 -- 172.21.15.40:0/3014482629 >> v1:172.21.15.62:6789/0 conn(0x7fa44406df40 legacy=0x7fa44413fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa4499e1640 1 -- 172.21.15.40:0/3014482629 >> v1:172.21.15.40:6789/0 conn(0x7fa44413e730 legacy=0x7fa444149b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:18.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa4499e1640 1 -- 172.21.15.40:0/3014482629 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa444071af0 con 0x7fa4440a79a0 2024-04-22T05:55:18.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa4499e1640 1 --2- 172.21.15.40:0/3014482629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4440a79a0 0x7fa4440a7d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa434009870 tx=0x7fa43401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8764c1f53262c63e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:18.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa4489df640 1 -- 172.21.15.40:0/3014482629 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4340ab600 con 0x7fa4440a79a0 2024-04-22T05:55:18.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa4489df640 1 -- 172.21.15.40:0/3014482629 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4340a7050 con 0x7fa4440a79a0 2024-04-22T05:55:18.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.910+0000 7fa4489df640 1 -- 172.21.15.40:0/3014482629 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4340ab980 con 0x7fa4440a79a0 2024-04-22T05:55:18.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 -- 172.21.15.40:0/3014482629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4440a79a0 msgr2=0x7fa4440a7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:18.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 --2- 172.21.15.40:0/3014482629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4440a79a0 0x7fa4440a7d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa434009870 tx=0x7fa43401c3e0 comp rx=0 tx=0).stop 2024-04-22T05:55:18.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 -- 172.21.15.40:0/3014482629 shutdown_connections 2024-04-22T05:55:18.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 --2- 172.21.15.40:0/3014482629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4440a79a0 0x7fa4440a7d80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:18.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 -- 172.21.15.40:0/3014482629 >> 172.21.15.40:0/3014482629 conn(0x7fa4440b3170 msgr2=0x7fa4440b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:18.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 -- 172.21.15.40:0/3014482629 shutdown_connections 2024-04-22T05:55:18.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 -- 172.21.15.40:0/3014482629 wait complete. 2024-04-22T05:55:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 Processor -- start 2024-04-22T05:55:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 -- start start 2024-04-22T05:55:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa44406df40 0x7fa4440ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4440a79a0 0x7fa4441399b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa44413e730 0x7fa444132d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4441350e0 con 0x7fa4440a79a0 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa444134740 con 0x7fa44413e730 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4441327d0 con 0x7fa44406df40 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa4499e1640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa44406df40 0x7fa4440ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa4491e0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4440a79a0 0x7fa4441399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa4499e1640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa44406df40 0x7fa4440ae980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:36106/0 (socket says 172.21.15.40:36106) 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa4499e1640 1 -- 172.21.15.40:0/4198389021 learned_addr learned my addr 172.21.15.40:0/4198389021 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44a1e2640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa44413e730 0x7fa444132d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa4491e0640 1 -- 172.21.15.40:0/4198389021 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa44406df40 msgr2=0x7fa4440ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa4491e0640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa44406df40 0x7fa4440ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa4491e0640 1 -- 172.21.15.40:0/4198389021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa44413e730 msgr2=0x7fa444132d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa4491e0640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa44413e730 0x7fa444132d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa4491e0640 1 -- 172.21.15.40:0/4198389021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4340094d0 con 0x7fa4440a79a0 2024-04-22T05:55:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa4499e1640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa44406df40 0x7fa4440ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa4491e0640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4440a79a0 0x7fa4441399b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa44000b3e0 tx=0x7fa44000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa43a7fc640 1 -- 172.21.15.40:0/4198389021 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa440011830 con 0x7fa4440a79a0 2024-04-22T05:55:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa43a7fc640 1 -- 172.21.15.40:0/4198389021 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa440011990 con 0x7fa4440a79a0 2024-04-22T05:55:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 -- 172.21.15.40:0/4198389021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4440714f0 con 0x7fa4440a79a0 2024-04-22T05:55:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa43a7fc640 1 -- 172.21.15.40:0/4198389021 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa440011d80 con 0x7fa4440a79a0 2024-04-22T05:55:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 -- 172.21.15.40:0/4198389021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44412d2c0 con 0x7fa4440a79a0 2024-04-22T05:55:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.914+0000 7fa44bc45640 1 -- 172.21.15.40:0/4198389021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa444067380 con 0x7fa4440a79a0 2024-04-22T05:55:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.918+0000 7fa43a7fc640 1 -- 172.21.15.40:0/4198389021 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa440004020 con 0x7fa4440a79a0 2024-04-22T05:55:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.918+0000 7fa43a7fc640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa430066690 0x7fa430068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.918+0000 7fa43a7fc640 1 -- 172.21.15.40:0/4198389021 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7fa440083d50 con 0x7fa4440a79a0 2024-04-22T05:55:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.918+0000 7fa4499e1640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa430066690 0x7fa430068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.918+0000 7fa4499e1640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa430066690 0x7fa430068b50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa434009870 tx=0x7fa4340a6000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:18.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:18.922+0000 7fa43a7fc640 1 -- 172.21.15.40:0/4198389021 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7fa440087030 con 0x7fa4440a79a0 2024-04-22T05:55:19.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.074+0000 7fa44bc45640 1 -- 172.21.15.40:0/4198389021 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa44406e590 con 0x7fa430066690 2024-04-22T05:55:19.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.090+0000 7fa43a7fc640 1 -- 172.21.15.40:0/4198389021 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+153354 (secure 0 0 0) 0x7fa4340ab870 con 0x7fa430066690 2024-04-22T05:55:19.093 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:55:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-22T05:55:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.094+0000 7fa44bc45640 1 -- 172.21.15.40:0/4198389021 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa430066690 msgr2=0x7fa430068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.094+0000 7fa44bc45640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa430066690 0x7fa430068b50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa434009870 tx=0x7fa4340a6000 comp rx=0 tx=0).stop 2024-04-22T05:55:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.094+0000 7fa44bc45640 1 -- 172.21.15.40:0/4198389021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4440a79a0 msgr2=0x7fa4441399b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.094+0000 7fa44bc45640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4440a79a0 0x7fa4441399b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa44000b3e0 tx=0x7fa44000b8a0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.094+0000 7fa44bc45640 1 -- 172.21.15.40:0/4198389021 shutdown_connections 2024-04-22T05:55:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.094+0000 7fa44bc45640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa430066690 0x7fa430068b50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.094+0000 7fa44bc45640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa44413e730 0x7fa444132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.094+0000 7fa44bc45640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4440a79a0 0x7fa4441399b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.094+0000 7fa44bc45640 1 --2- 172.21.15.40:0/4198389021 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa44406df40 0x7fa4440ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.094+0000 7fa44bc45640 1 -- 172.21.15.40:0/4198389021 >> 172.21.15.40:0/4198389021 conn(0x7fa4440b3170 msgr2=0x7fa44406ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.094+0000 7fa44bc45640 1 -- 172.21.15.40:0/4198389021 shutdown_connections 2024-04-22T05:55:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.094+0000 7fa44bc45640 1 -- 172.21.15.40:0/4198389021 wait complete. 2024-04-22T05:55:19.114 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-04-22T05:55:19.024834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3568104744,"num_objects":880,"num_object_clones":0,"num_object_copies":1760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":2651,"num_read_kb":2646,"num_write":157290,"num_write_kb":5628649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158620,"ondisk_log_size":158620,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9131676,"kb_used_data":6638684,"kb_used_omap":12,"kb_used_meta":2492915,"kb_avail":745843044,"statfs":{"total":773094113280,"available":763743277056,"internally_reserved":0,"allocated":6798012416,"data_stored":6835380396,"data_compressed":194286840,"data_compressed_allocated":202113024,"data_compressed_original":244916224,"omap_allocated":12744,"internal_metadata":2552745528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32768,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":12,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003234"},"pg_stats":[{"pgid":"3.1f","version":"21'3322","reported_seq":3348,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.159313+0000","last_change":"2024-04-22T05:50:47.121259+0000","last_active":"2024-04-22T05:54:02.159313+0000","last_peered":"2024-04-22T05:54:02.159313+0000","last_clean":"2024-04-22T05:54:02.159313+0000","last_became_active":"2024-04-22T05:50:47.120501+0000","last_became_peered":"2024-04-22T05:50:47.120501+0000","last_unstale":"2024-04-22T05:54:02.159313+0000","last_undegraded":"2024-04-22T05:54:02.159313+0000","last_fullsized":"2024-04-22T05:54:02.159313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3322,"log_dups_size":0,"ondisk_log_size":3322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:03:25.965635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109323296,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":3347,"num_write_kb":168857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"22'5496","reported_seq":5517,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:09.741154+0000","last_change":"2024-04-22T05:50:47.118054+0000","last_active":"2024-04-22T05:54:09.741154+0000","last_peered":"2024-04-22T05:54:09.741154+0000","last_clean":"2024-04-22T05:54:09.741154+0000","last_became_active":"2024-04-22T05:50:47.117274+0000","last_became_peered":"2024-04-22T05:50:47.117274+0000","last_unstale":"2024-04-22T05:54:09.741154+0000","last_undegraded":"2024-04-22T05:54:09.741154+0000","last_fullsized":"2024-04-22T05:54:09.741154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5496,"log_dups_size":0,"ondisk_log_size":5496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:26.168980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125960704,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":28,"num_read_kb":84,"num_write":5488,"num_write_kb":185156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"22'3417","reported_seq":3440,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:15.141589+0000","last_change":"2024-04-22T05:50:47.122061+0000","last_active":"2024-04-22T05:54:15.141589+0000","last_peered":"2024-04-22T05:54:15.141589+0000","last_clean":"2024-04-22T05:54:15.141589+0000","last_became_active":"2024-04-22T05:50:47.120601+0000","last_became_peered":"2024-04-22T05:50:47.120601+0000","last_unstale":"2024-04-22T05:54:15.141589+0000","last_undegraded":"2024-04-22T05:54:15.141589+0000","last_fullsized":"2024-04-22T05:54:15.141589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3417,"log_dups_size":0,"ondisk_log_size":3417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:30:19.535589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72812200,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":3406,"num_write_kb":94339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2981","reported_seq":2993,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-22T05:53:19.072284+0000","last_change":"2024-04-22T05:50:47.117092+0000","last_active":"2024-04-22T05:53:19.072284+0000","last_peered":"2024-04-22T05:53:19.072284+0000","last_clean":"2024-04-22T05:53:19.072284+0000","last_became_active":"2024-04-22T05:50:47.115803+0000","last_became_peered":"2024-04-22T05:50:47.115803+0000","last_unstale":"2024-04-22T05:53:19.072284+0000","last_undegraded":"2024-04-22T05:53:19.072284+0000","last_fullsized":"2024-04-22T05:53:19.072284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":2981,"log_dups_size":0,"ondisk_log_size":2981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:40:33.403824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89522176,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2976,"num_write_kb":149441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"22'5220","reported_seq":5255,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.593346+0000","last_change":"2024-04-22T05:50:47.124002+0000","last_active":"2024-04-22T05:54:02.593346+0000","last_peered":"2024-04-22T05:54:02.593346+0000","last_clean":"2024-04-22T05:54:02.593346+0000","last_became_active":"2024-04-22T05:50:47.123073+0000","last_became_peered":"2024-04-22T05:50:47.123073+0000","last_unstale":"2024-04-22T05:54:02.593346+0000","last_undegraded":"2024-04-22T05:54:02.593346+0000","last_fullsized":"2024-04-22T05:54:02.593346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5220,"log_dups_size":0,"ondisk_log_size":5220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:02:52.999238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150994944,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":90,"num_read_kb":78,"num_write":5209,"num_write_kb":201924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"22'6502","reported_seq":6522,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:10.515152+0000","last_change":"2024-04-22T05:50:47.118816+0000","last_active":"2024-04-22T05:54:10.515152+0000","last_peered":"2024-04-22T05:54:10.515152+0000","last_clean":"2024-04-22T05:54:10.515152+0000","last_became_active":"2024-04-22T05:50:47.118038+0000","last_became_peered":"2024-04-22T05:50:47.118038+0000","last_unstale":"2024-04-22T05:54:10.515152+0000","last_undegraded":"2024-04-22T05:54:10.515152+0000","last_fullsized":"2024-04-22T05:54:10.515152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:16:18.439809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113311744,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":6483,"num_write_kb":139343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"22'4753","reported_seq":4765,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:20.636889+0000","last_change":"2024-04-22T05:50:47.114124+0000","last_active":"2024-04-22T05:54:20.636889+0000","last_peered":"2024-04-22T05:54:20.636889+0000","last_clean":"2024-04-22T05:54:20.636889+0000","last_became_active":"2024-04-22T05:50:47.113368+0000","last_became_peered":"2024-04-22T05:50:47.113368+0000","last_unstale":"2024-04-22T05:54:20.636889+0000","last_undegraded":"2024-04-22T05:54:20.636889+0000","last_fullsized":"2024-04-22T05:54:20.636889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4753,"log_dups_size":0,"ondisk_log_size":4753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:26:25.586314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112566272,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":123245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"22'8053","reported_seq":8072,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:21.044772+0000","last_change":"2024-04-22T05:50:47.114390+0000","last_active":"2024-04-22T05:54:21.044772+0000","last_peered":"2024-04-22T05:54:21.044772+0000","last_clean":"2024-04-22T05:54:21.044772+0000","last_became_active":"2024-04-22T05:50:47.113569+0000","last_became_peered":"2024-04-22T05:50:47.113569+0000","last_unstale":"2024-04-22T05:54:21.044772+0000","last_undegraded":"2024-04-22T05:54:21.044772+0000","last_fullsized":"2024-04-22T05:54:21.044772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":8053,"log_dups_size":0,"ondisk_log_size":8053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:18:39.555599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":8065,"num_write_kb":185808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"22'6472","reported_seq":6495,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:55:07.378031+0000","last_change":"2024-04-22T05:50:47.122110+0000","last_active":"2024-04-22T05:55:07.378031+0000","last_peered":"2024-04-22T05:55:07.378031+0000","last_clean":"2024-04-22T05:55:07.378031+0000","last_became_active":"2024-04-22T05:50:47.121332+0000","last_became_peered":"2024-04-22T05:50:47.121332+0000","last_unstale":"2024-04-22T05:55:07.378031+0000","last_undegraded":"2024-04-22T05:55:07.378031+0000","last_fullsized":"2024-04-22T05:55:07.378031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6472,"log_dups_size":0,"ondisk_log_size":6472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:41.522456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111009792,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":6495,"num_write_kb":211703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"22'3761","reported_seq":3775,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.474346+0000","last_change":"2024-04-22T05:50:47.120024+0000","last_active":"2024-04-22T05:54:02.474346+0000","last_peered":"2024-04-22T05:54:02.474346+0000","last_clean":"2024-04-22T05:54:02.474346+0000","last_became_active":"2024-04-22T05:50:47.119031+0000","last_became_peered":"2024-04-22T05:50:47.119031+0000","last_unstale":"2024-04-22T05:54:02.474346+0000","last_undegraded":"2024-04-22T05:54:02.474346+0000","last_fullsized":"2024-04-22T05:54:02.474346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3761,"log_dups_size":0,"ondisk_log_size":3761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:17:48.991277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":3747,"num_write_kb":173061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"22'7962","reported_seq":7975,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:19.971677+0000","last_change":"2024-04-22T05:50:47.119142+0000","last_active":"2024-04-22T05:54:19.971677+0000","last_peered":"2024-04-22T05:54:19.971677+0000","last_clean":"2024-04-22T05:54:19.971677+0000","last_became_active":"2024-04-22T05:50:47.118399+0000","last_became_peered":"2024-04-22T05:50:47.118399+0000","last_unstale":"2024-04-22T05:54:19.971677+0000","last_undegraded":"2024-04-22T05:54:19.971677+0000","last_fullsized":"2024-04-22T05:54:19.971677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":7962,"log_dups_size":0,"ondisk_log_size":7962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:42:14.831153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7958,"num_write_kb":234882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"22'7046","reported_seq":7087,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:13.904714+0000","last_change":"2024-04-22T05:50:47.131544+0000","last_active":"2024-04-22T05:54:13.904714+0000","last_peered":"2024-04-22T05:54:13.904714+0000","last_clean":"2024-04-22T05:54:13.904714+0000","last_became_active":"2024-04-22T05:50:47.130842+0000","last_became_peered":"2024-04-22T05:50:47.130842+0000","last_unstale":"2024-04-22T05:54:13.904714+0000","last_undegraded":"2024-04-22T05:54:13.904714+0000","last_fullsized":"2024-04-22T05:54:13.904714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:14:10.141463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149815296,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":118,"num_read_kb":94,"num_write":7055,"num_write_kb":232332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.083465+0000","last_change":"2024-04-22T05:50:41.050585+0000","last_active":"2024-04-22T05:54:02.083465+0000","last_peered":"2024-04-22T05:54:02.083465+0000","last_clean":"2024-04-22T05:54:02.083465+0000","last_became_active":"2024-04-22T05:50:41.049696+0000","last_became_peered":"2024-04-22T05:50:41.049696+0000","last_unstale":"2024-04-22T05:54:02.083465+0000","last_undegraded":"2024-04-22T05:54:02.083465+0000","last_fullsized":"2024-04-22T05:54:02.083465+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:00:56.754144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'4443","reported_seq":4476,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:17.298588+0000","last_change":"2024-04-22T05:50:47.114576+0000","last_active":"2024-04-22T05:54:17.298588+0000","last_peered":"2024-04-22T05:54:17.298588+0000","last_clean":"2024-04-22T05:54:17.298588+0000","last_became_active":"2024-04-22T05:50:47.113838+0000","last_became_peered":"2024-04-22T05:50:47.113838+0000","last_unstale":"2024-04-22T05:54:17.298588+0000","last_undegraded":"2024-04-22T05:54:17.298588+0000","last_fullsized":"2024-04-22T05:54:17.298588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4443,"log_dups_size":0,"ondisk_log_size":4443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:29:28.189329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120194896,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":4449,"num_write_kb":155490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.160033+0000","last_change":"2024-04-22T05:50:41.048959+0000","last_active":"2024-04-22T05:54:02.160033+0000","last_peered":"2024-04-22T05:54:02.160033+0000","last_clean":"2024-04-22T05:54:02.160033+0000","last_became_active":"2024-04-22T05:50:41.048628+0000","last_became_peered":"2024-04-22T05:50:41.048628+0000","last_unstale":"2024-04-22T05:54:02.160033+0000","last_undegraded":"2024-04-22T05:54:02.160033+0000","last_fullsized":"2024-04-22T05:54:02.160033+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:46:57.799120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'3252","reported_seq":3267,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.591342+0000","last_change":"2024-04-22T05:50:47.130110+0000","last_active":"2024-04-22T05:54:02.591342+0000","last_peered":"2024-04-22T05:54:02.591342+0000","last_clean":"2024-04-22T05:54:02.591342+0000","last_became_active":"2024-04-22T05:50:47.129322+0000","last_became_peered":"2024-04-22T05:50:47.129322+0000","last_unstale":"2024-04-22T05:54:02.591342+0000","last_undegraded":"2024-04-22T05:54:02.591342+0000","last_fullsized":"2024-04-22T05:54:02.591342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3252,"log_dups_size":0,"ondisk_log_size":3252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:10:00.803610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":3214,"num_write_kb":155139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:53.173287+0000","last_change":"2024-04-22T05:50:41.341624+0000","last_active":"2024-04-22T05:54:53.173287+0000","last_peered":"2024-04-22T05:54:53.173287+0000","last_clean":"2024-04-22T05:54:53.173287+0000","last_became_active":"2024-04-22T05:50:41.340643+0000","last_became_peered":"2024-04-22T05:50:41.340643+0000","last_unstale":"2024-04-22T05:54:53.173287+0000","last_undegraded":"2024-04-22T05:54:53.173287+0000","last_fullsized":"2024-04-22T05:54:53.173287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:47:12.897727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'6361","reported_seq":6380,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:22.636657+0000","last_change":"2024-04-22T05:50:47.172954+0000","last_active":"2024-04-22T05:54:22.636657+0000","last_peered":"2024-04-22T05:54:22.636657+0000","last_clean":"2024-04-22T05:54:22.636657+0000","last_became_active":"2024-04-22T05:50:47.172121+0000","last_became_peered":"2024-04-22T05:50:47.172121+0000","last_unstale":"2024-04-22T05:54:22.636657+0000","last_undegraded":"2024-04-22T05:54:22.636657+0000","last_fullsized":"2024-04-22T05:54:22.636657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6361,"log_dups_size":0,"ondisk_log_size":6361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:17:03.619291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":26,"num_read_kb":18,"num_write":6371,"num_write_kb":148611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.159809+0000","last_change":"2024-04-22T05:50:41.341548+0000","last_active":"2024-04-22T05:54:02.159809+0000","last_peered":"2024-04-22T05:54:02.159809+0000","last_clean":"2024-04-22T05:54:02.159809+0000","last_became_active":"2024-04-22T05:50:41.341254+0000","last_became_peered":"2024-04-22T05:50:41.341254+0000","last_unstale":"2024-04-22T05:54:02.159809+0000","last_undegraded":"2024-04-22T05:54:02.159809+0000","last_fullsized":"2024-04-22T05:54:02.159809+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:34:01.082016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"21'3305","reported_seq":3329,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.174237+0000","last_change":"2024-04-22T05:50:47.121899+0000","last_active":"2024-04-22T05:54:02.174237+0000","last_peered":"2024-04-22T05:54:02.174237+0000","last_clean":"2024-04-22T05:54:02.174237+0000","last_became_active":"2024-04-22T05:50:47.120072+0000","last_became_peered":"2024-04-22T05:50:47.120072+0000","last_unstale":"2024-04-22T05:54:02.174237+0000","last_undegraded":"2024-04-22T05:54:02.174237+0000","last_fullsized":"2024-04-22T05:54:02.174237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3305,"log_dups_size":0,"ondisk_log_size":3305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:05:35.227673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99223376,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":3325,"num_write_kb":171289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'3629","reported_seq":3651,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.096142+0000","last_change":"2024-04-22T05:50:47.125635+0000","last_active":"2024-04-22T05:54:02.096142+0000","last_peered":"2024-04-22T05:54:02.096142+0000","last_clean":"2024-04-22T05:54:02.096142+0000","last_became_active":"2024-04-22T05:50:47.124841+0000","last_became_peered":"2024-04-22T05:50:47.124841+0000","last_unstale":"2024-04-22T05:54:02.096142+0000","last_undegraded":"2024-04-22T05:54:02.096142+0000","last_fullsized":"2024-04-22T05:54:02.096142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3629,"log_dups_size":0,"ondisk_log_size":3629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:08:34.775098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":3649,"num_write_kb":178556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.171665+0000","last_change":"2024-04-22T05:50:41.051239+0000","last_active":"2024-04-22T05:54:02.171665+0000","last_peered":"2024-04-22T05:54:02.171665+0000","last_clean":"2024-04-22T05:54:02.171665+0000","last_became_active":"2024-04-22T05:50:41.050711+0000","last_became_peered":"2024-04-22T05:50:41.050711+0000","last_unstale":"2024-04-22T05:54:02.171665+0000","last_undegraded":"2024-04-22T05:54:02.171665+0000","last_fullsized":"2024-04-22T05:54:02.171665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:07:59.435066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"22'5388","reported_seq":5404,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.590950+0000","last_change":"2024-04-22T05:50:47.173005+0000","last_active":"2024-04-22T05:54:02.590950+0000","last_peered":"2024-04-22T05:54:02.590950+0000","last_clean":"2024-04-22T05:54:02.590950+0000","last_became_active":"2024-04-22T05:50:47.172218+0000","last_became_peered":"2024-04-22T05:50:47.172218+0000","last_unstale":"2024-04-22T05:54:02.590950+0000","last_undegraded":"2024-04-22T05:54:02.590950+0000","last_fullsized":"2024-04-22T05:54:02.590950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5388,"log_dups_size":0,"ondisk_log_size":5388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:49:38.626473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138609688,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":5358,"num_write_kb":177059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.080259+0000","last_change":"2024-04-22T05:50:41.051710+0000","last_active":"2024-04-22T05:54:02.080259+0000","last_peered":"2024-04-22T05:54:02.080259+0000","last_clean":"2024-04-22T05:54:02.080259+0000","last_became_active":"2024-04-22T05:50:41.051259+0000","last_became_peered":"2024-04-22T05:50:41.051259+0000","last_unstale":"2024-04-22T05:54:02.080259+0000","last_undegraded":"2024-04-22T05:54:02.080259+0000","last_fullsized":"2024-04-22T05:54:02.080259+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:41:46.406807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"22'4949","reported_seq":5063,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.592758+0000","last_change":"2024-04-22T05:50:47.120166+0000","last_active":"2024-04-22T05:54:02.592758+0000","last_peered":"2024-04-22T05:54:02.592758+0000","last_clean":"2024-04-22T05:54:02.592758+0000","last_became_active":"2024-04-22T05:50:47.119414+0000","last_became_peered":"2024-04-22T05:50:47.119414+0000","last_unstale":"2024-04-22T05:54:02.592758+0000","last_undegraded":"2024-04-22T05:54:02.592758+0000","last_fullsized":"2024-04-22T05:54:02.592758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:46:02.360307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124520800,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":4927,"num_write_kb":205619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:43.171572+0000","last_change":"2024-04-22T05:50:41.045192+0000","last_active":"2024-04-22T05:54:43.171572+0000","last_peered":"2024-04-22T05:54:43.171572+0000","last_clean":"2024-04-22T05:54:43.171572+0000","last_became_active":"2024-04-22T05:50:41.044865+0000","last_became_peered":"2024-04-22T05:50:41.044865+0000","last_unstale":"2024-04-22T05:54:43.171572+0000","last_undegraded":"2024-04-22T05:54:43.171572+0000","last_fullsized":"2024-04-22T05:54:43.171572+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:37:43.523824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"22'5431","reported_seq":5445,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:17.517971+0000","last_change":"2024-04-22T05:50:47.114255+0000","last_active":"2024-04-22T05:54:17.517971+0000","last_peered":"2024-04-22T05:54:17.517971+0000","last_clean":"2024-04-22T05:54:17.517971+0000","last_became_active":"2024-04-22T05:50:47.113405+0000","last_became_peered":"2024-04-22T05:50:47.113405+0000","last_unstale":"2024-04-22T05:54:17.517971+0000","last_undegraded":"2024-04-22T05:54:17.517971+0000","last_fullsized":"2024-04-22T05:54:17.517971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:16.829289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128647688,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5434,"num_write_kb":203340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.171887+0000","last_change":"2024-04-22T05:50:41.052651+0000","last_active":"2024-04-22T05:54:02.171887+0000","last_peered":"2024-04-22T05:54:02.171887+0000","last_clean":"2024-04-22T05:54:02.171887+0000","last_became_active":"2024-04-22T05:50:41.052349+0000","last_became_peered":"2024-04-22T05:50:41.052349+0000","last_unstale":"2024-04-22T05:54:02.171887+0000","last_undegraded":"2024-04-22T05:54:02.171887+0000","last_fullsized":"2024-04-22T05:54:02.171887+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:46:51.494717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:43.171606+0000","last_change":"2024-04-22T05:50:40.047233+0000","last_active":"2024-04-22T05:54:43.171606+0000","last_peered":"2024-04-22T05:54:43.171606+0000","last_clean":"2024-04-22T05:54:43.171606+0000","last_became_active":"2024-04-22T05:50:40.046329+0000","last_became_peered":"2024-04-22T05:50:40.046329+0000","last_unstale":"2024-04-22T05:54:43.171606+0000","last_undegraded":"2024-04-22T05:54:43.171606+0000","last_fullsized":"2024-04-22T05:54:43.171606+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:39.011105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:39.011105+0000","last_clean_scrub_stamp":"2024-04-22T05:50:39.011105+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:16:02.587654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"22'3929","reported_seq":3945,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.591331+0000","last_change":"2024-04-22T05:50:47.116127+0000","last_active":"2024-04-22T05:54:02.591331+0000","last_peered":"2024-04-22T05:54:02.591331+0000","last_clean":"2024-04-22T05:54:02.591331+0000","last_became_active":"2024-04-22T05:50:47.115449+0000","last_became_peered":"2024-04-22T05:50:47.115449+0000","last_unstale":"2024-04-22T05:54:02.591331+0000","last_undegraded":"2024-04-22T05:54:02.591331+0000","last_fullsized":"2024-04-22T05:54:02.591331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:33.600047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":3867,"num_write_kb":173187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"22'10215","reported_seq":10334,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:55:07.352677+0000","last_change":"2024-04-22T05:50:47.173998+0000","last_active":"2024-04-22T05:55:07.352677+0000","last_peered":"2024-04-22T05:55:07.352677+0000","last_clean":"2024-04-22T05:55:07.352677+0000","last_became_active":"2024-04-22T05:50:47.173440+0000","last_became_peered":"2024-04-22T05:50:47.173440+0000","last_unstale":"2024-04-22T05:55:07.352677+0000","last_undegraded":"2024-04-22T05:55:07.352677+0000","last_fullsized":"2024-04-22T05:55:07.352677+0000","mapping_epoch":19,"log_start":"21'200","ondisk_log_start":"21'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:40.518919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":356,"num_read_kb":316,"num_write":8415,"num_write_kb":237085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"22'5035","reported_seq":5048,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:22.686699+0000","last_change":"2024-04-22T05:50:47.121542+0000","last_active":"2024-04-22T05:54:22.686699+0000","last_peered":"2024-04-22T05:54:22.686699+0000","last_clean":"2024-04-22T05:54:22.686699+0000","last_became_active":"2024-04-22T05:50:47.120541+0000","last_became_peered":"2024-04-22T05:50:47.120541+0000","last_unstale":"2024-04-22T05:54:22.686699+0000","last_undegraded":"2024-04-22T05:54:22.686699+0000","last_fullsized":"2024-04-22T05:54:22.686699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5035,"log_dups_size":0,"ondisk_log_size":5035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:07:12.244690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5009,"num_write_kb":193539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"21'4714","reported_seq":4730,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.160520+0000","last_change":"2024-04-22T05:50:47.123811+0000","last_active":"2024-04-22T05:54:02.160520+0000","last_peered":"2024-04-22T05:54:02.160520+0000","last_clean":"2024-04-22T05:54:02.160520+0000","last_became_active":"2024-04-22T05:50:47.123115+0000","last_became_peered":"2024-04-22T05:50:47.123115+0000","last_unstale":"2024-04-22T05:54:02.160520+0000","last_undegraded":"2024-04-22T05:54:02.160520+0000","last_fullsized":"2024-04-22T05:54:02.160520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:05:32.458383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600592,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":4720,"num_write_kb":218508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"22'4376","reported_seq":4392,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.593981+0000","last_change":"2024-04-22T05:50:47.116574+0000","last_active":"2024-04-22T05:54:02.593981+0000","last_peered":"2024-04-22T05:54:02.593981+0000","last_clean":"2024-04-22T05:54:02.593981+0000","last_became_active":"2024-04-22T05:50:47.114834+0000","last_became_peered":"2024-04-22T05:50:47.114834+0000","last_unstale":"2024-04-22T05:54:02.593981+0000","last_undegraded":"2024-04-22T05:54:02.593981+0000","last_fullsized":"2024-04-22T05:54:02.593981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4376,"log_dups_size":0,"ondisk_log_size":4376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:37:17.012651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":4338,"num_write_kb":218182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'3554","reported_seq":3567,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.101329+0000","last_change":"2024-04-22T05:50:47.117704+0000","last_active":"2024-04-22T05:55:18.101329+0000","last_peered":"2024-04-22T05:55:18.101329+0000","last_clean":"2024-04-22T05:55:18.101329+0000","last_became_active":"2024-04-22T05:50:47.113260+0000","last_became_peered":"2024-04-22T05:50:47.113260+0000","last_unstale":"2024-04-22T05:55:18.101329+0000","last_undegraded":"2024-04-22T05:55:18.101329+0000","last_fullsized":"2024-04-22T05:55:18.101329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:54:55.343145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104202240,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3525,"num_write_kb":160132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'3630","reported_seq":3736,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.179885+0000","last_change":"2024-04-22T05:50:47.123901+0000","last_active":"2024-04-22T05:55:18.179885+0000","last_peered":"2024-04-22T05:55:18.179885+0000","last_clean":"2024-04-22T05:55:18.179885+0000","last_became_active":"2024-04-22T05:50:47.122822+0000","last_became_peered":"2024-04-22T05:50:47.122822+0000","last_unstale":"2024-04-22T05:55:18.179885+0000","last_undegraded":"2024-04-22T05:55:18.179885+0000","last_fullsized":"2024-04-22T05:55:18.179885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3630,"log_dups_size":0,"ondisk_log_size":3630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:05.611537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83894800,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":300,"num_read_kb":235,"num_write":3880,"num_write_kb":108782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'4331","reported_seq":4367,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.101140+0000","last_change":"2024-04-22T05:50:47.118515+0000","last_active":"2024-04-22T05:55:18.101140+0000","last_peered":"2024-04-22T05:55:18.101140+0000","last_clean":"2024-04-22T05:55:18.101140+0000","last_became_active":"2024-04-22T05:50:47.117734+0000","last_became_peered":"2024-04-22T05:50:47.117734+0000","last_unstale":"2024-04-22T05:55:18.101140+0000","last_undegraded":"2024-04-22T05:55:18.101140+0000","last_fullsized":"2024-04-22T05:55:18.101140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:33:43.659487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126091776,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":92,"num_read_kb":134,"num_write":4330,"num_write_kb":222405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"22'3195","reported_seq":3207,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.583389+0000","last_change":"2024-04-22T05:50:47.125698+0000","last_active":"2024-04-22T05:54:02.583389+0000","last_peered":"2024-04-22T05:54:02.583389+0000","last_clean":"2024-04-22T05:54:02.583389+0000","last_became_active":"2024-04-22T05:50:47.124711+0000","last_became_peered":"2024-04-22T05:50:47.124711+0000","last_unstale":"2024-04-22T05:54:02.583389+0000","last_undegraded":"2024-04-22T05:54:02.583389+0000","last_fullsized":"2024-04-22T05:54:02.583389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3195,"log_dups_size":0,"ondisk_log_size":3195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:06:16.139033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3149,"num_write_kb":120072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"21'3697","reported_seq":4016,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.080934+0000","last_change":"2024-04-22T05:50:47.116263+0000","last_active":"2024-04-22T05:54:02.080934+0000","last_peered":"2024-04-22T05:54:02.080934+0000","last_clean":"2024-04-22T05:54:02.080934+0000","last_became_active":"2024-04-22T05:50:47.115535+0000","last_became_peered":"2024-04-22T05:50:47.115535+0000","last_unstale":"2024-04-22T05:54:02.080934+0000","last_undegraded":"2024-04-22T05:54:02.080934+0000","last_fullsized":"2024-04-22T05:54:02.080934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:20.378220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92406272,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1026,"num_read_kb":919,"num_write":3912,"num_write_kb":173447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"22'3436","reported_seq":3451,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.588199+0000","last_change":"2024-04-22T05:50:47.129955+0000","last_active":"2024-04-22T05:54:02.588199+0000","last_peered":"2024-04-22T05:54:02.588199+0000","last_clean":"2024-04-22T05:54:02.588199+0000","last_became_active":"2024-04-22T05:50:47.129345+0000","last_became_peered":"2024-04-22T05:50:47.129345+0000","last_unstale":"2024-04-22T05:54:02.588199+0000","last_undegraded":"2024-04-22T05:54:02.588199+0000","last_fullsized":"2024-04-22T05:54:02.588199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3436,"log_dups_size":0,"ondisk_log_size":3436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:40.789469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":71,"num_write":3499,"num_write_kb":148421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"22'6848","reported_seq":6862,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:15.867722+0000","last_change":"2024-04-22T05:50:47.127371+0000","last_active":"2024-04-22T05:54:15.867722+0000","last_peered":"2024-04-22T05:54:15.867722+0000","last_clean":"2024-04-22T05:54:15.867722+0000","last_became_active":"2024-04-22T05:50:47.126563+0000","last_became_peered":"2024-04-22T05:50:47.126563+0000","last_unstale":"2024-04-22T05:54:15.867722+0000","last_undegraded":"2024-04-22T05:54:15.867722+0000","last_fullsized":"2024-04-22T05:54:15.867722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6848,"log_dups_size":0,"ondisk_log_size":6848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:16:53.013583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1,"num_read_kb":64,"num_write":6824,"num_write_kb":157279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3567514376,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":2535,"num_read_kb":2546,"num_write":157131,"num_write_kb":5626233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6796107776,"data_stored":6833678488,"data_compressed":194275682,"data_compressed_allocated":202047488,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":158503,"ondisk_log_size":158503,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11158,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":14,"seq":60129542201,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591948,"kb_used_data":408260,"kb_used_omap":1,"kb_used_meta":183678,"kb_avail":93779892,"statfs":{"total":96636764160,"available":96030609408,"internally_reserved":0,"allocated":418058240,"data_stored":419207031,"data_compressed":12754442,"data_compressed_allocated":13279232,"data_compressed_original":14893056,"omap_allocated":1594,"internal_metadata":188086726},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.84599999999999997,"15min":0.78800000000000003},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.95799999999999996,"5min":20.890000000000001,"15min":20.890000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.622,"15min":0.66000000000000003},"min":{"1min":0.22600000000000001,"5min":0.185,"15min":0.185},"max":{"1min":1.032,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.76100000000000001}]},{"osd":1,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66400000000000003,"15min":0.67200000000000004},"min":{"1min":0.24199999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.221,"15min":0.221},"max":{"1min":1.04,"5min":1.218,"15min":1.218},"last":0.83099999999999996}]},{"osd":3,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.747,"15min":0.86499999999999999},"min":{"1min":0.26100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.92700000000000005,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.82099999999999995,"15min":0.88700000000000001},"min":{"1min":0.33000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":1.111,"5min":3.27,"15min":3.27},"last":0.91600000000000004}]},{"osd":4,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81599999999999995,"15min":0.80200000000000005},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.879},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.86299999999999999,"15min":0.85499999999999998},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.27,"5min":1.54,"15min":1.54},"last":1.1000000000000001}]},{"osd":5,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82999999999999996,"15min":0.84599999999999997},"min":{"1min":0.39100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.389,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.36699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.6439999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.94099999999999995}]},{"osd":6,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.92800000000000005,"15min":0.95399999999999996},"min":{"1min":0.41699999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.204,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94499999999999995,"15min":0.97399999999999998},"min":{"1min":0.39400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.597,"5min":1.651,"15min":1.651},"last":1.0660000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.2869999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.879,"5min":1.0920000000000001,"15min":1.02},"min":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4750000000000001,"5min":17.995999999999999,"15min":17.995999999999999},"last":1.024}]}]},{"osd":6,"up_from":14,"seq":60129542204,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418576,"kb_used_data":311176,"kb_used_omap":1,"kb_used_meta":107390,"kb_avail":93953264,"statfs":{"total":96636764160,"available":96208142336,"internally_reserved":0,"allocated":318644224,"data_stored":318792523,"data_compressed":4811634,"data_compressed_allocated":5001216,"data_compressed_original":5484544,"omap_allocated":1594,"internal_metadata":109967814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":1.506,"15min":1.8380000000000001},"min":{"1min":0.373,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.0720000000000001,"5min":34.100000000000001,"15min":34.100000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":1.0589999999999999,"15min":0.85199999999999998},"min":{"1min":0.439,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.131,"5min":30.306000000000001,"15min":30.306000000000001},"last":1.026}]},{"osd":1,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.79200000000000004,"15min":0.73799999999999999},"min":{"1min":0.47799999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.2849999999999999,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.107},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.81599999999999995,"15min":0.77600000000000002},"min":{"1min":0.32600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":1.2430000000000001,"5min":1.74,"15min":1.74},"last":1.3089999999999999}]},{"osd":2,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.88500000000000001,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.216,"15min":0.216},"max":{"1min":1.3009999999999999,"5min":2.488,"15min":2.488},"last":1.212},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.92700000000000005,"15min":0.88700000000000001},"min":{"1min":0.503,"5min":0.31,"15min":0.31},"max":{"1min":1.2370000000000001,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":1.1850000000000001}]},{"osd":3,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84799999999999998,"15min":0.86099999999999999},"min":{"1min":0.56499999999999995,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.101,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.88},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0209999999999999,"15min":0.95999999999999996},"min":{"1min":0.45200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.149,"5min":13.539,"15min":13.539},"last":1.056}]},{"osd":4,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.89600000000000002,"15min":0.92900000000000005},"min":{"1min":0.60199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.242,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88500000000000001,"15min":0.86699999999999999},"min":{"1min":0.501,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.204,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.139}]},{"osd":5,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.101,"15min":1.2310000000000001},"min":{"1min":0.66700000000000004,"5min":0.312,"15min":0.312},"max":{"1min":1.1970000000000001,"5min":11.922000000000001,"15min":11.922000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1,"15min":0.995},"min":{"1min":0.46999999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3080000000000001,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.71899999999999997}]},{"osd":7,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0269999999999999,"15min":0.995},"min":{"1min":0.60599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.353,"5min":2.129,"15min":2.129},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.101,"15min":1.101},"min":{"1min":0.67300000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3879999999999999,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.96399999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542202,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1090848,"kb_used_data":843224,"kb_used_omap":1,"kb_used_meta":247614,"kb_avail":93280992,"statfs":{"total":96636764160,"available":95519735808,"internally_reserved":0,"allocated":863461376,"data_stored":869115087,"data_compressed":21037680,"data_compressed_allocated":21954560,"data_compressed_original":28217344,"omap_allocated":1594,"internal_metadata":253557190},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76400000000000001,"15min":0.68999999999999995},"min":{"1min":0.443,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.3919999999999999,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.875,"15min":0.91000000000000003},"min":{"1min":0.29299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.3300000000000001,"5min":4.6390000000000002,"15min":4.6390000000000002},"last":0.69199999999999995}]},{"osd":1,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.78900000000000003,"15min":0.71399999999999997},"min":{"1min":0.35799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":1.3500000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.129},{"interface":"front","average":{"1min":0.74199999999999999,"5min":1.0049999999999999,"15min":1.181},"min":{"1min":0.376,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.0449999999999999,"5min":8.202,"15min":8.202},"last":0.72199999999999998}]},{"osd":2,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.83299999999999996,"15min":0.78800000000000003},"min":{"1min":0.39900000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4299999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.90300000000000002,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.474,"5min":1.673,"15min":1.673},"last":1.1850000000000001}]},{"osd":3,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":2.0110000000000001,"15min":2.8210000000000002},"min":{"1min":0.33600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.423,"5min":44.186999999999998,"15min":44.186999999999998},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.081,"15min":1.198},"min":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1419999999999999,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.99199999999999999}]},{"osd":4,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.1659999999999999,"15min":1.3240000000000001},"min":{"1min":0.29899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.3520000000000001,"5min":11.438000000000001,"15min":11.438000000000001},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.98599999999999999,"15min":0.95999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.171,"5min":2.181,"15min":2.181},"last":0.84499999999999997}]},{"osd":6,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":0.93500000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1830000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":1.097,"15min":1.335},"min":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.222,"5min":11.474,"15min":11.474},"last":1.0129999999999999}]},{"osd":7,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.107,"15min":1.081},"min":{"1min":0.38200000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.542,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.01,"5min":1.1100000000000001,"15min":1.1579999999999999},"min":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3009999999999999,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.038}]}]},{"osd":4,"up_from":14,"seq":60129542203,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1920752,"kb_used_data":1416424,"kb_used_omap":1,"kb_used_meta":504318,"kb_avail":92451088,"statfs":{"total":96636764160,"available":94669914112,"internally_reserved":0,"allocated":1450418176,"data_stored":1463130155,"data_compressed":51243088,"data_compressed_allocated":53432320,"data_compressed_original":67096576,"omap_allocated":1594,"internal_metadata":516422086},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":0.91800000000000004,"15min":0.75800000000000001},"min":{"1min":0.28499999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":12.618,"5min":12.618,"15min":12.618},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.0409999999999999,"15min":0.92000000000000004},"min":{"1min":0.35699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":12.548,"5min":12.548,"15min":12.548},"last":0.78600000000000003}]},{"osd":1,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.751,"15min":0.755},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.1970000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.5509999999999999,"5min":0.96099999999999997,"15min":0.81799999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":12.449999999999999,"5min":12.449999999999999,"15min":12.449999999999999},"last":1.1910000000000001}]},{"osd":2,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.33900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.097,"5min":2.226,"15min":2.226},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.048,"15min":0.92800000000000005},"min":{"1min":0.42899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":12.673,"5min":12.673,"15min":12.673},"last":1.0569999999999999}]},{"osd":3,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.90400000000000003,"15min":0.88400000000000001},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.2070000000000001,"5min":3.5329999999999999,"15min":3.5329999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.41899999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.335,"5min":2.254,"15min":2.254},"last":1.012}]},{"osd":5,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.1040000000000001,"15min":1.0349999999999999},"min":{"1min":0.40200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":12.721,"5min":12.721,"15min":12.721},"last":1.25},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.95999999999999996},"min":{"1min":0.39500000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.369,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.96199999999999997}]},{"osd":6,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.1919999999999999,"15min":1.04},"min":{"1min":0.55600000000000005,"5min":0.312,"15min":0.312},"max":{"1min":12.765000000000001,"5min":12.765000000000001,"15min":12.765000000000001},"last":1.117},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0860000000000001,"15min":1.087},"min":{"1min":0.57699999999999996,"5min":0.372,"15min":0.372},"max":{"1min":1.431,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.302}]},{"osd":7,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.038,"15min":1.0569999999999999},"min":{"1min":0.51800000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.456,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.091,"15min":1.0629999999999999},"min":{"1min":0.505,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.6120000000000001,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.278}]}]},{"osd":0,"up_from":14,"seq":60129542203,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1378064,"kb_used_data":1017992,"kb_used_omap":1,"kb_used_meta":360062,"kb_avail":92993776,"statfs":{"total":96636764160,"available":95225626624,"internally_reserved":0,"allocated":1042423808,"data_stored":1047499431,"data_compressed":29839922,"data_compressed_allocated":30982144,"data_compressed_original":36835328,"omap_allocated":1593,"internal_metadata":368703943},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60599999999999998,"15min":0.55000000000000004},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.792,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.74199999999999999,"15min":0.80600000000000005},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1.4770000000000001,"5min":4.7489999999999997,"15min":4.7489999999999997},"last":0.76600000000000001}]},{"osd":2,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.68700000000000006,"15min":0.625},"min":{"1min":0.28399999999999997,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.90100000000000002},"min":{"1min":0.33200000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.6579999999999999,"5min":4.7880000000000003,"15min":4.7880000000000003},"last":1.05}]},{"osd":3,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.5429999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.79400000000000004,"15min":0.76300000000000001},"min":{"1min":0.315,"5min":0.307,"15min":0.307},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.97799999999999998}]},{"osd":4,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.01,"15min":1.085},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.232,"5min":4.8440000000000003,"15min":4.8440000000000003},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.91400000000000003,"15min":0.90400000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.258,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.177}]},{"osd":5,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.94799999999999995,"15min":0.93200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.47,"5min":2.722,"15min":2.722},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.0289999999999999,"15min":1.107},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.474,"5min":4.8860000000000001,"15min":4.8860000000000001},"last":1.127}]},{"osd":6,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.96199999999999997,"15min":0.88900000000000001},"min":{"1min":0.40000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.5449999999999999,"5min":1.76,"15min":1.76},"last":1.3160000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.94199999999999995,"15min":0.89300000000000002},"min":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3560000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.3700000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2629999999999999,"15min":1.4019999999999999},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.4980000000000002,"5min":8.6839999999999993,"15min":8.6839999999999993},"last":1.224},{"interface":"front","average":{"1min":1.304,"5min":1.1970000000000001,"15min":1.2509999999999999},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.4649999999999999,"5min":4.4649999999999999,"15min":4.4649999999999999},"last":1.4770000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1329560,"kb_used_data":959696,"kb_used_omap":1,"kb_used_meta":369854,"kb_avail":93042280,"statfs":{"total":96636764160,"available":95275294720,"internally_reserved":0,"allocated":982728704,"data_stored":987917207,"data_compressed":23928438,"data_compressed_allocated":24997888,"data_compressed_original":30998528,"omap_allocated":1589,"internal_metadata":378730955},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":1.3640000000000001,"15min":1.0680000000000001},"min":{"1min":0.45700000000000002,"5min":0.224,"15min":0.224},"max":{"1min":1.1839999999999999,"5min":46.598999999999997,"15min":46.598999999999997},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":1.0309999999999999,"15min":1.2709999999999999},"min":{"1min":0.32700000000000001,"5min":0.216,"15min":0.216},"max":{"1min":1.141,"5min":15.144,"15min":15.144},"last":1.099}]},{"osd":1,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.80900000000000005,"15min":0.77500000000000002},"min":{"1min":0.35699999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.347,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.018,"5min":0.89700000000000002,"15min":0.82699999999999996},"min":{"1min":0.48599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3300000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.2310000000000001}]},{"osd":2,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.89500000000000002,"15min":0.871},"min":{"1min":0.33000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.262,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.94899999999999995,"15min":0.873},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4419999999999999,"5min":3.387,"15min":3.387},"last":1.006}]},{"osd":3,"last update":"Mon Apr 22 05:54:50 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.999,"15min":0.85899999999999999},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3520000000000001,"5min":7.1699999999999999,"15min":7.1699999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.89600000000000002,"15min":0.82799999999999996},"min":{"1min":0.45600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.399,"5min":3.6299999999999999,"15min":3.6299999999999999},"last":1.2729999999999999}]},{"osd":4,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.258,"15min":1.2909999999999999},"min":{"1min":0.34399999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.4650000000000001,"5min":9.6620000000000008,"15min":9.6620000000000008},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.94799999999999995,"15min":0.90200000000000002},"min":{"1min":0.40600000000000003,"5min":0.252,"15min":0.252},"max":{"1min":1.335,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.05}]},{"osd":5,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.95399999999999996,"15min":0.89900000000000002},"min":{"1min":0.29299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.54,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.208},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1659999999999999,"15min":0.997},"min":{"1min":0.39100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.72,"5min":7.2430000000000003,"15min":7.2430000000000003},"last":1.3759999999999999}]},{"osd":6,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.04,"15min":0.95599999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5649999999999999,"5min":3.5939999999999999,"15min":3.5939999999999999},"last":1.143},{"interface":"front","average":{"1min":1.208,"5min":1.2450000000000001,"15min":1.0649999999999999},"min":{"1min":0.53400000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.794,"5min":7.2960000000000003,"15min":7.2960000000000003},"last":1.5640000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1412644,"kb_used_data":1016540,"kb_used_omap":1,"kb_used_meta":396094,"kb_avail":92959196,"statfs":{"total":96636764160,"available":95190216704,"internally_reserved":0,"allocated":1040936960,"data_stored":1044385363,"data_compressed":24230251,"data_compressed_allocated":25128960,"data_compressed_original":29396992,"omap_allocated":1592,"internal_metadata":405600712},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.69199999999999995,"15min":0.82399999999999995},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":1.079,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.63},{"interface":"front","average":{"1min":0.66000000000000003,"5min":1.4970000000000001,"15min":2.2440000000000002},"min":{"1min":0.28799999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.129,"5min":22.721,"15min":22.721},"last":1.24}]},{"osd":1,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.80700000000000005,"15min":0.95099999999999996},"min":{"1min":0.23999999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.5840000000000001,"5min":9.6560000000000006,"15min":9.6560000000000006},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.3100000000000001,"5min":2.028,"15min":2.028},"last":0.69799999999999995}]},{"osd":2,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":1.3009999999999999,"15min":1.619},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.3029999999999999,"5min":22.763999999999999,"15min":22.763999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.94299999999999995,"15min":1.0920000000000001},"min":{"1min":0.313,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.6190000000000002,"5min":9.7739999999999991,"15min":9.7739999999999991},"last":0.75700000000000001}]},{"osd":4,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.7270000000000001,"15min":1.8879999999999999},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.657,"5min":27.384,"15min":27.384},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.571,"15min":1.931},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.508,"5min":22.824000000000002,"15min":22.824000000000002},"last":0.92100000000000004}]},{"osd":5,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.762,"15min":2.3220000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.4039999999999999,"5min":25.565999999999999,"15min":25.565999999999999},"last":1.139},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.98799999999999999,"15min":0.93999999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.6829999999999998,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":1.2010000000000001}]},{"osd":6,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.6419999999999999,"15min":1.9570000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.706,"5min":22.794,"15min":22.794},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.006,"5min":1.171,"15min":1.323},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6240000000000001,"5min":9.9559999999999995,"15min":9.9559999999999995},"last":1.3180000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":2.4089999999999998,"15min":3.1589999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.796,"5min":36.770000000000003,"15min":36.770000000000003},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.4039999999999999,"15min":1.4059999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.9100000000000001,"5min":16.239000000000001,"15min":16.239000000000001},"last":1.4019999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":989284,"kb_used_data":665372,"kb_used_omap":1,"kb_used_meta":323902,"kb_avail":93382556,"statfs":{"total":96636764160,"available":95623737344,"internally_reserved":0,"allocated":681340928,"data_stored":685333599,"data_compressed":26441385,"data_compressed_allocated":27336704,"data_compressed_original":31993856,"omap_allocated":1594,"internal_metadata":331676102},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.66600000000000004,"15min":0.70699999999999996},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.83099999999999996,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.26200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.1679999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.79600000000000004}]},{"osd":2,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.26100000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.014,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.68600000000000005,"15min":0.61799999999999999},"min":{"1min":0.34599999999999997,"5min":0.22,"15min":0.22},"max":{"1min":1.0900000000000001,"5min":3.871,"15min":3.871},"last":1.0189999999999999}]},{"osd":3,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.29699999999999999,"5min":0.252,"15min":0.252},"max":{"1min":1.1970000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.083},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.83499999999999996,"15min":0.85199999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.256,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.877}]},{"osd":4,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.0409999999999999,"15min":1.1180000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.242,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.044,"15min":1.044},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":2.0880000000000001,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.2370000000000001}]},{"osd":5,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.98299999999999998,"15min":0.94499999999999995},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.2650000000000001,"5min":3.8460000000000001,"15min":3.8460000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.98799999999999999,"15min":0.96999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.325,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.173}]},{"osd":6,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97699999999999998,"15min":0.92300000000000004},"min":{"1min":0.38400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3999999999999999,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.268},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.98799999999999999,"15min":0.95799999999999996},"min":{"1min":0.42699999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.222}]},{"osd":7,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.042,"15min":1.004},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.1800000000000002,"5min":3.8159999999999998,"15min":3.8159999999999998},"last":1.369},{"interface":"front","average":{"1min":1.091,"5min":1.0720000000000001,"15min":1.0369999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.615,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.3500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1042149376,"data_stored":1046890840,"data_compressed":29834343,"data_compressed_allocated":30949376,"data_compressed_original":36323328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":681115648,"data_stored":685252972,"data_compressed":26441385,"data_compressed_allocated":27336704,"data_compressed_original":31993856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417832960,"data_stored":419126404,"data_compressed":12754442,"data_compressed_allocated":13279232,"data_compressed_original":14893056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1040711680,"data_stored":1044304736,"data_compressed":24230251,"data_compressed_allocated":25128960,"data_compressed_original":29396992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1450192896,"data_stored":1463049528,"data_compressed":51243088,"data_compressed_allocated":53432320,"data_compressed_original":67096576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":863236096,"data_stored":869034460,"data_compressed":21037680,"data_compressed_allocated":21954560,"data_compressed_original":28217344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":318418944,"data_stored":318711896,"data_compressed":4811634,"data_compressed_allocated":5001216,"data_compressed_original":5484544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":982450176,"data_stored":987307652,"data_compressed":23922859,"data_compressed_allocated":24965120,"data_compressed_original":30486528,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T05:55:19.118 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T05:55:19.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd7f8f640 1 Processor -- start 2024-04-22T05:55:19.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd7f8f640 1 -- start start 2024-04-22T05:55:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd7f8f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2fd006e1a0 0x7f2fd013b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd7f8f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd0071d00 con 0x7f2fd013b760 2024-04-22T05:55:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd7f8f640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd006e580 con 0x7f2fd00a7920 2024-04-22T05:55:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd7f8f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd006f8e0 con 0x7f2fd006e1a0 2024-04-22T05:55:19.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd5d2b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2fd006e1a0 0x7f2fd013b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd652c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2fd013b760 0x7f2fd013ea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38560/0 (socket says 172.21.15.40:38560) 2024-04-22T05:55:19.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd652c640 1 -- 172.21.15.40:0/4208810584 learned_addr learned my addr 172.21.15.40:0/4208810584 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:55:19.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd5d2b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2fd006e1a0 0x7f2fd013b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47040/0 (socket says 172.21.15.40:47040) 2024-04-22T05:55:19.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd5d2b640 1 -- 172.21.15.40:0/4208810584 >> v1:172.21.15.62:6789/0 conn(0x7f2fd00a7920 legacy=0x7f2fd013e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd5d2b640 1 -- 172.21.15.40:0/4208810584 >> v1:172.21.15.40:6789/0 conn(0x7f2fd013b760 legacy=0x7f2fd013ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd4d29640 1 -- 172.21.15.40:0/4208810584 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982296996 0 0) 0x7f2fd0071d00 con 0x7f2fd013b760 2024-04-22T05:55:19.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.238+0000 7f2fd5d2b640 1 -- 172.21.15.40:0/4208810584 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd00a8f60 con 0x7f2fd006e1a0 2024-04-22T05:55:19.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd5d2b640 1 --2- 172.21.15.40:0/4208810584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd006e1a0 0x7f2fd013b220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00ba60 tx=0x7f2fcc01ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f79f4302135d9df server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:19.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd4d29640 1 -- 172.21.15.40:0/4208810584 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0a7910 con 0x7f2fd006e1a0 2024-04-22T05:55:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd4d29640 1 -- 172.21.15.40:0/4208810584 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc0a7a70 con 0x7f2fd006e1a0 2024-04-22T05:55:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd4d29640 1 -- 172.21.15.40:0/4208810584 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0a7e00 con 0x7f2fd006e1a0 2024-04-22T05:55:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/4208810584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd006e1a0 msgr2=0x7f2fd013b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 --2- 172.21.15.40:0/4208810584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd006e1a0 0x7f2fd013b220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00ba60 tx=0x7f2fcc01ea30 comp rx=0 tx=0).stop 2024-04-22T05:55:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/4208810584 shutdown_connections 2024-04-22T05:55:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 --2- 172.21.15.40:0/4208810584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd006e1a0 0x7f2fd013b220 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/4208810584 >> 172.21.15.40:0/4208810584 conn(0x7f2fd00b3170 msgr2=0x7f2fd00b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/4208810584 shutdown_connections 2024-04-22T05:55:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/4208810584 wait complete. 2024-04-22T05:55:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 Processor -- start 2024-04-22T05:55:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 -- start start 2024-04-22T05:55:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd006e1a0 0x7f2fd012d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fd00a7920 0x7f2fd0130cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2fd013b760 0x7f2fd01d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd0131590 con 0x7f2fd006e1a0 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd0131a20 con 0x7f2fd00a7920 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd0131200 con 0x7f2fd013b760 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd5d2b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd006e1a0 0x7f2fd012d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd652c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2fd013b760 0x7f2fd01d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd5d2b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd006e1a0 0x7f2fd012d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47044/0 (socket says 172.21.15.40:47044) 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd5d2b640 1 -- 172.21.15.40:0/2344941998 learned_addr learned my addr 172.21.15.40:0/2344941998 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd552a640 1 --2- 172.21.15.40:0/2344941998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fd00a7920 0x7f2fd0130cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd5d2b640 1 -- 172.21.15.40:0/2344941998 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2fd013b760 msgr2=0x7f2fd01d80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd5d2b640 1 --2- 172.21.15.40:0/2344941998 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2fd013b760 0x7f2fd01d80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd5d2b640 1 -- 172.21.15.40:0/2344941998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fd00a7920 msgr2=0x7f2fd0130cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd5d2b640 1 --2- 172.21.15.40:0/2344941998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fd00a7920 0x7f2fd0130cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd5d2b640 1 -- 172.21.15.40:0/2344941998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc00b6c0 con 0x7f2fd006e1a0 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd5d2b640 1 --2- 172.21.15.40:0/2344941998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd006e1a0 0x7f2fd012d870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc01c660 tx=0x7f2fcc01b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fc67fc640 1 -- 172.21.15.40:0/2344941998 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc004230 con 0x7f2fd006e1a0 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fc67fc640 1 -- 172.21.15.40:0/2344941998 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc004390 con 0x7f2fd006e1a0 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/2344941998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd01302d0 con 0x7f2fd006e1a0 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fc67fc640 1 -- 172.21.15.40:0/2344941998 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0af930 con 0x7f2fd006e1a0 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.242+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/2344941998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd01d87e0 con 0x7f2fd006e1a0 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.246+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/2344941998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd0067380 con 0x7f2fd006e1a0 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.246+0000 7f2fc67fc640 1 -- 172.21.15.40:0/2344941998 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2fcc0afa90 con 0x7f2fd006e1a0 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.246+0000 7f2fc67fc640 1 --2- 172.21.15.40:0/2344941998 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f2fb4066690 0x7f2fb4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.246+0000 7f2fc67fc640 1 -- 172.21.15.40:0/2344941998 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f2fcc121870 con 0x7f2fd006e1a0 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.246+0000 7f2fd552a640 1 --2- 172.21.15.40:0/2344941998 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f2fb4066690 0x7f2fb4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.246+0000 7f2fd552a640 1 --2- 172.21.15.40:0/2344941998 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f2fb4066690 0x7f2fb4068b50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0132100 tx=0x7f2fc0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.250+0000 7f2fc67fc640 1 -- 172.21.15.40:0/2344941998 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f2fcc0eee40 con 0x7f2fd006e1a0 2024-04-22T05:55:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.406+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/2344941998 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2fd01316d0 con 0x7f2fb4066690 2024-04-22T05:55:19.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.418+0000 7f2fc67fc640 1 -- 172.21.15.40:0/2344941998 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+153354 (secure 0 0 0) 0x7f2fc0001380 con 0x7f2fb4066690 2024-04-22T05:55:19.422 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:55:19.423 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-22T05:55:19.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.422+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/2344941998 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f2fb4066690 msgr2=0x7f2fb4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.422+0000 7f2fd7f8f640 1 --2- 172.21.15.40:0/2344941998 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f2fb4066690 0x7f2fb4068b50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0132100 tx=0x7f2fc0009040 comp rx=0 tx=0).stop 2024-04-22T05:55:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.422+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/2344941998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd006e1a0 msgr2=0x7f2fd012d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.422+0000 7f2fd7f8f640 1 --2- 172.21.15.40:0/2344941998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd006e1a0 0x7f2fd012d870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc01c660 tx=0x7f2fcc01b7d0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.422+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/2344941998 shutdown_connections 2024-04-22T05:55:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.422+0000 7f2fd7f8f640 1 --2- 172.21.15.40:0/2344941998 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f2fb4066690 0x7f2fb4068b50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.422+0000 7f2fd7f8f640 1 --2- 172.21.15.40:0/2344941998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fd00a7920 0x7f2fd0130cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.422+0000 7f2fd7f8f640 1 --2- 172.21.15.40:0/2344941998 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f2fd013b760 0x7f2fd01d80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.422+0000 7f2fd7f8f640 1 --2- 172.21.15.40:0/2344941998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd006e1a0 0x7f2fd012d870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.422+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/2344941998 >> 172.21.15.40:0/2344941998 conn(0x7f2fd00b3170 msgr2=0x7f2fd00b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.422+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/2344941998 shutdown_connections 2024-04-22T05:55:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.422+0000 7f2fd7f8f640 1 -- 172.21.15.40:0/2344941998 wait complete. 2024-04-22T05:55:19.441 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-04-22T05:55:19.024834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3568104744,"num_objects":880,"num_object_clones":0,"num_object_copies":1760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":2651,"num_read_kb":2646,"num_write":157290,"num_write_kb":5628649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158620,"ondisk_log_size":158620,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9131676,"kb_used_data":6638684,"kb_used_omap":12,"kb_used_meta":2492915,"kb_avail":745843044,"statfs":{"total":773094113280,"available":763743277056,"internally_reserved":0,"allocated":6798012416,"data_stored":6835380396,"data_compressed":194286840,"data_compressed_allocated":202113024,"data_compressed_original":244916224,"omap_allocated":12744,"internal_metadata":2552745528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32768,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":12,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003234"},"pg_stats":[{"pgid":"3.1f","version":"21'3322","reported_seq":3348,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.159313+0000","last_change":"2024-04-22T05:50:47.121259+0000","last_active":"2024-04-22T05:54:02.159313+0000","last_peered":"2024-04-22T05:54:02.159313+0000","last_clean":"2024-04-22T05:54:02.159313+0000","last_became_active":"2024-04-22T05:50:47.120501+0000","last_became_peered":"2024-04-22T05:50:47.120501+0000","last_unstale":"2024-04-22T05:54:02.159313+0000","last_undegraded":"2024-04-22T05:54:02.159313+0000","last_fullsized":"2024-04-22T05:54:02.159313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3322,"log_dups_size":0,"ondisk_log_size":3322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:03:25.965635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109323296,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":3347,"num_write_kb":168857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"22'5496","reported_seq":5517,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:09.741154+0000","last_change":"2024-04-22T05:50:47.118054+0000","last_active":"2024-04-22T05:54:09.741154+0000","last_peered":"2024-04-22T05:54:09.741154+0000","last_clean":"2024-04-22T05:54:09.741154+0000","last_became_active":"2024-04-22T05:50:47.117274+0000","last_became_peered":"2024-04-22T05:50:47.117274+0000","last_unstale":"2024-04-22T05:54:09.741154+0000","last_undegraded":"2024-04-22T05:54:09.741154+0000","last_fullsized":"2024-04-22T05:54:09.741154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5496,"log_dups_size":0,"ondisk_log_size":5496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:26.168980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125960704,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":28,"num_read_kb":84,"num_write":5488,"num_write_kb":185156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"22'3417","reported_seq":3440,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:15.141589+0000","last_change":"2024-04-22T05:50:47.122061+0000","last_active":"2024-04-22T05:54:15.141589+0000","last_peered":"2024-04-22T05:54:15.141589+0000","last_clean":"2024-04-22T05:54:15.141589+0000","last_became_active":"2024-04-22T05:50:47.120601+0000","last_became_peered":"2024-04-22T05:50:47.120601+0000","last_unstale":"2024-04-22T05:54:15.141589+0000","last_undegraded":"2024-04-22T05:54:15.141589+0000","last_fullsized":"2024-04-22T05:54:15.141589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3417,"log_dups_size":0,"ondisk_log_size":3417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:30:19.535589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72812200,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":3406,"num_write_kb":94339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2981","reported_seq":2993,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-22T05:53:19.072284+0000","last_change":"2024-04-22T05:50:47.117092+0000","last_active":"2024-04-22T05:53:19.072284+0000","last_peered":"2024-04-22T05:53:19.072284+0000","last_clean":"2024-04-22T05:53:19.072284+0000","last_became_active":"2024-04-22T05:50:47.115803+0000","last_became_peered":"2024-04-22T05:50:47.115803+0000","last_unstale":"2024-04-22T05:53:19.072284+0000","last_undegraded":"2024-04-22T05:53:19.072284+0000","last_fullsized":"2024-04-22T05:53:19.072284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":2981,"log_dups_size":0,"ondisk_log_size":2981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:40:33.403824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89522176,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2976,"num_write_kb":149441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"22'5220","reported_seq":5255,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.593346+0000","last_change":"2024-04-22T05:50:47.124002+0000","last_active":"2024-04-22T05:54:02.593346+0000","last_peered":"2024-04-22T05:54:02.593346+0000","last_clean":"2024-04-22T05:54:02.593346+0000","last_became_active":"2024-04-22T05:50:47.123073+0000","last_became_peered":"2024-04-22T05:50:47.123073+0000","last_unstale":"2024-04-22T05:54:02.593346+0000","last_undegraded":"2024-04-22T05:54:02.593346+0000","last_fullsized":"2024-04-22T05:54:02.593346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5220,"log_dups_size":0,"ondisk_log_size":5220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:02:52.999238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150994944,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":90,"num_read_kb":78,"num_write":5209,"num_write_kb":201924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"22'6502","reported_seq":6522,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:10.515152+0000","last_change":"2024-04-22T05:50:47.118816+0000","last_active":"2024-04-22T05:54:10.515152+0000","last_peered":"2024-04-22T05:54:10.515152+0000","last_clean":"2024-04-22T05:54:10.515152+0000","last_became_active":"2024-04-22T05:50:47.118038+0000","last_became_peered":"2024-04-22T05:50:47.118038+0000","last_unstale":"2024-04-22T05:54:10.515152+0000","last_undegraded":"2024-04-22T05:54:10.515152+0000","last_fullsized":"2024-04-22T05:54:10.515152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:16:18.439809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113311744,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":6483,"num_write_kb":139343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"22'4753","reported_seq":4765,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:20.636889+0000","last_change":"2024-04-22T05:50:47.114124+0000","last_active":"2024-04-22T05:54:20.636889+0000","last_peered":"2024-04-22T05:54:20.636889+0000","last_clean":"2024-04-22T05:54:20.636889+0000","last_became_active":"2024-04-22T05:50:47.113368+0000","last_became_peered":"2024-04-22T05:50:47.113368+0000","last_unstale":"2024-04-22T05:54:20.636889+0000","last_undegraded":"2024-04-22T05:54:20.636889+0000","last_fullsized":"2024-04-22T05:54:20.636889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4753,"log_dups_size":0,"ondisk_log_size":4753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:26:25.586314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112566272,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":123245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"22'8053","reported_seq":8072,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:21.044772+0000","last_change":"2024-04-22T05:50:47.114390+0000","last_active":"2024-04-22T05:54:21.044772+0000","last_peered":"2024-04-22T05:54:21.044772+0000","last_clean":"2024-04-22T05:54:21.044772+0000","last_became_active":"2024-04-22T05:50:47.113569+0000","last_became_peered":"2024-04-22T05:50:47.113569+0000","last_unstale":"2024-04-22T05:54:21.044772+0000","last_undegraded":"2024-04-22T05:54:21.044772+0000","last_fullsized":"2024-04-22T05:54:21.044772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":8053,"log_dups_size":0,"ondisk_log_size":8053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:18:39.555599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":8065,"num_write_kb":185808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"22'6472","reported_seq":6495,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:55:07.378031+0000","last_change":"2024-04-22T05:50:47.122110+0000","last_active":"2024-04-22T05:55:07.378031+0000","last_peered":"2024-04-22T05:55:07.378031+0000","last_clean":"2024-04-22T05:55:07.378031+0000","last_became_active":"2024-04-22T05:50:47.121332+0000","last_became_peered":"2024-04-22T05:50:47.121332+0000","last_unstale":"2024-04-22T05:55:07.378031+0000","last_undegraded":"2024-04-22T05:55:07.378031+0000","last_fullsized":"2024-04-22T05:55:07.378031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6472,"log_dups_size":0,"ondisk_log_size":6472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:41.522456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111009792,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":6495,"num_write_kb":211703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"22'3761","reported_seq":3775,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.474346+0000","last_change":"2024-04-22T05:50:47.120024+0000","last_active":"2024-04-22T05:54:02.474346+0000","last_peered":"2024-04-22T05:54:02.474346+0000","last_clean":"2024-04-22T05:54:02.474346+0000","last_became_active":"2024-04-22T05:50:47.119031+0000","last_became_peered":"2024-04-22T05:50:47.119031+0000","last_unstale":"2024-04-22T05:54:02.474346+0000","last_undegraded":"2024-04-22T05:54:02.474346+0000","last_fullsized":"2024-04-22T05:54:02.474346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3761,"log_dups_size":0,"ondisk_log_size":3761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:17:48.991277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":3747,"num_write_kb":173061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"22'7962","reported_seq":7975,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:19.971677+0000","last_change":"2024-04-22T05:50:47.119142+0000","last_active":"2024-04-22T05:54:19.971677+0000","last_peered":"2024-04-22T05:54:19.971677+0000","last_clean":"2024-04-22T05:54:19.971677+0000","last_became_active":"2024-04-22T05:50:47.118399+0000","last_became_peered":"2024-04-22T05:50:47.118399+0000","last_unstale":"2024-04-22T05:54:19.971677+0000","last_undegraded":"2024-04-22T05:54:19.971677+0000","last_fullsized":"2024-04-22T05:54:19.971677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":7962,"log_dups_size":0,"ondisk_log_size":7962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:42:14.831153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7958,"num_write_kb":234882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"22'7046","reported_seq":7087,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:13.904714+0000","last_change":"2024-04-22T05:50:47.131544+0000","last_active":"2024-04-22T05:54:13.904714+0000","last_peered":"2024-04-22T05:54:13.904714+0000","last_clean":"2024-04-22T05:54:13.904714+0000","last_became_active":"2024-04-22T05:50:47.130842+0000","last_became_peered":"2024-04-22T05:50:47.130842+0000","last_unstale":"2024-04-22T05:54:13.904714+0000","last_undegraded":"2024-04-22T05:54:13.904714+0000","last_fullsized":"2024-04-22T05:54:13.904714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:14:10.141463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149815296,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":118,"num_read_kb":94,"num_write":7055,"num_write_kb":232332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.083465+0000","last_change":"2024-04-22T05:50:41.050585+0000","last_active":"2024-04-22T05:54:02.083465+0000","last_peered":"2024-04-22T05:54:02.083465+0000","last_clean":"2024-04-22T05:54:02.083465+0000","last_became_active":"2024-04-22T05:50:41.049696+0000","last_became_peered":"2024-04-22T05:50:41.049696+0000","last_unstale":"2024-04-22T05:54:02.083465+0000","last_undegraded":"2024-04-22T05:54:02.083465+0000","last_fullsized":"2024-04-22T05:54:02.083465+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:00:56.754144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'4443","reported_seq":4476,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:17.298588+0000","last_change":"2024-04-22T05:50:47.114576+0000","last_active":"2024-04-22T05:54:17.298588+0000","last_peered":"2024-04-22T05:54:17.298588+0000","last_clean":"2024-04-22T05:54:17.298588+0000","last_became_active":"2024-04-22T05:50:47.113838+0000","last_became_peered":"2024-04-22T05:50:47.113838+0000","last_unstale":"2024-04-22T05:54:17.298588+0000","last_undegraded":"2024-04-22T05:54:17.298588+0000","last_fullsized":"2024-04-22T05:54:17.298588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4443,"log_dups_size":0,"ondisk_log_size":4443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:29:28.189329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120194896,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":4449,"num_write_kb":155490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.160033+0000","last_change":"2024-04-22T05:50:41.048959+0000","last_active":"2024-04-22T05:54:02.160033+0000","last_peered":"2024-04-22T05:54:02.160033+0000","last_clean":"2024-04-22T05:54:02.160033+0000","last_became_active":"2024-04-22T05:50:41.048628+0000","last_became_peered":"2024-04-22T05:50:41.048628+0000","last_unstale":"2024-04-22T05:54:02.160033+0000","last_undegraded":"2024-04-22T05:54:02.160033+0000","last_fullsized":"2024-04-22T05:54:02.160033+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:46:57.799120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'3252","reported_seq":3267,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.591342+0000","last_change":"2024-04-22T05:50:47.130110+0000","last_active":"2024-04-22T05:54:02.591342+0000","last_peered":"2024-04-22T05:54:02.591342+0000","last_clean":"2024-04-22T05:54:02.591342+0000","last_became_active":"2024-04-22T05:50:47.129322+0000","last_became_peered":"2024-04-22T05:50:47.129322+0000","last_unstale":"2024-04-22T05:54:02.591342+0000","last_undegraded":"2024-04-22T05:54:02.591342+0000","last_fullsized":"2024-04-22T05:54:02.591342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3252,"log_dups_size":0,"ondisk_log_size":3252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:10:00.803610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":3214,"num_write_kb":155139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:53.173287+0000","last_change":"2024-04-22T05:50:41.341624+0000","last_active":"2024-04-22T05:54:53.173287+0000","last_peered":"2024-04-22T05:54:53.173287+0000","last_clean":"2024-04-22T05:54:53.173287+0000","last_became_active":"2024-04-22T05:50:41.340643+0000","last_became_peered":"2024-04-22T05:50:41.340643+0000","last_unstale":"2024-04-22T05:54:53.173287+0000","last_undegraded":"2024-04-22T05:54:53.173287+0000","last_fullsized":"2024-04-22T05:54:53.173287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:47:12.897727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'6361","reported_seq":6380,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:22.636657+0000","last_change":"2024-04-22T05:50:47.172954+0000","last_active":"2024-04-22T05:54:22.636657+0000","last_peered":"2024-04-22T05:54:22.636657+0000","last_clean":"2024-04-22T05:54:22.636657+0000","last_became_active":"2024-04-22T05:50:47.172121+0000","last_became_peered":"2024-04-22T05:50:47.172121+0000","last_unstale":"2024-04-22T05:54:22.636657+0000","last_undegraded":"2024-04-22T05:54:22.636657+0000","last_fullsized":"2024-04-22T05:54:22.636657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6361,"log_dups_size":0,"ondisk_log_size":6361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:17:03.619291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":26,"num_read_kb":18,"num_write":6371,"num_write_kb":148611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.159809+0000","last_change":"2024-04-22T05:50:41.341548+0000","last_active":"2024-04-22T05:54:02.159809+0000","last_peered":"2024-04-22T05:54:02.159809+0000","last_clean":"2024-04-22T05:54:02.159809+0000","last_became_active":"2024-04-22T05:50:41.341254+0000","last_became_peered":"2024-04-22T05:50:41.341254+0000","last_unstale":"2024-04-22T05:54:02.159809+0000","last_undegraded":"2024-04-22T05:54:02.159809+0000","last_fullsized":"2024-04-22T05:54:02.159809+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:34:01.082016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"21'3305","reported_seq":3329,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.174237+0000","last_change":"2024-04-22T05:50:47.121899+0000","last_active":"2024-04-22T05:54:02.174237+0000","last_peered":"2024-04-22T05:54:02.174237+0000","last_clean":"2024-04-22T05:54:02.174237+0000","last_became_active":"2024-04-22T05:50:47.120072+0000","last_became_peered":"2024-04-22T05:50:47.120072+0000","last_unstale":"2024-04-22T05:54:02.174237+0000","last_undegraded":"2024-04-22T05:54:02.174237+0000","last_fullsized":"2024-04-22T05:54:02.174237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3305,"log_dups_size":0,"ondisk_log_size":3305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:05:35.227673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99223376,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":3325,"num_write_kb":171289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'3629","reported_seq":3651,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.096142+0000","last_change":"2024-04-22T05:50:47.125635+0000","last_active":"2024-04-22T05:54:02.096142+0000","last_peered":"2024-04-22T05:54:02.096142+0000","last_clean":"2024-04-22T05:54:02.096142+0000","last_became_active":"2024-04-22T05:50:47.124841+0000","last_became_peered":"2024-04-22T05:50:47.124841+0000","last_unstale":"2024-04-22T05:54:02.096142+0000","last_undegraded":"2024-04-22T05:54:02.096142+0000","last_fullsized":"2024-04-22T05:54:02.096142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3629,"log_dups_size":0,"ondisk_log_size":3629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:08:34.775098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":3649,"num_write_kb":178556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.171665+0000","last_change":"2024-04-22T05:50:41.051239+0000","last_active":"2024-04-22T05:54:02.171665+0000","last_peered":"2024-04-22T05:54:02.171665+0000","last_clean":"2024-04-22T05:54:02.171665+0000","last_became_active":"2024-04-22T05:50:41.050711+0000","last_became_peered":"2024-04-22T05:50:41.050711+0000","last_unstale":"2024-04-22T05:54:02.171665+0000","last_undegraded":"2024-04-22T05:54:02.171665+0000","last_fullsized":"2024-04-22T05:54:02.171665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:07:59.435066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"22'5388","reported_seq":5404,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.590950+0000","last_change":"2024-04-22T05:50:47.173005+0000","last_active":"2024-04-22T05:54:02.590950+0000","last_peered":"2024-04-22T05:54:02.590950+0000","last_clean":"2024-04-22T05:54:02.590950+0000","last_became_active":"2024-04-22T05:50:47.172218+0000","last_became_peered":"2024-04-22T05:50:47.172218+0000","last_unstale":"2024-04-22T05:54:02.590950+0000","last_undegraded":"2024-04-22T05:54:02.590950+0000","last_fullsized":"2024-04-22T05:54:02.590950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5388,"log_dups_size":0,"ondisk_log_size":5388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:49:38.626473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138609688,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":5358,"num_write_kb":177059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.080259+0000","last_change":"2024-04-22T05:50:41.051710+0000","last_active":"2024-04-22T05:54:02.080259+0000","last_peered":"2024-04-22T05:54:02.080259+0000","last_clean":"2024-04-22T05:54:02.080259+0000","last_became_active":"2024-04-22T05:50:41.051259+0000","last_became_peered":"2024-04-22T05:50:41.051259+0000","last_unstale":"2024-04-22T05:54:02.080259+0000","last_undegraded":"2024-04-22T05:54:02.080259+0000","last_fullsized":"2024-04-22T05:54:02.080259+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:41:46.406807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"22'4949","reported_seq":5063,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.592758+0000","last_change":"2024-04-22T05:50:47.120166+0000","last_active":"2024-04-22T05:54:02.592758+0000","last_peered":"2024-04-22T05:54:02.592758+0000","last_clean":"2024-04-22T05:54:02.592758+0000","last_became_active":"2024-04-22T05:50:47.119414+0000","last_became_peered":"2024-04-22T05:50:47.119414+0000","last_unstale":"2024-04-22T05:54:02.592758+0000","last_undegraded":"2024-04-22T05:54:02.592758+0000","last_fullsized":"2024-04-22T05:54:02.592758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:46:02.360307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124520800,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":4927,"num_write_kb":205619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:43.171572+0000","last_change":"2024-04-22T05:50:41.045192+0000","last_active":"2024-04-22T05:54:43.171572+0000","last_peered":"2024-04-22T05:54:43.171572+0000","last_clean":"2024-04-22T05:54:43.171572+0000","last_became_active":"2024-04-22T05:50:41.044865+0000","last_became_peered":"2024-04-22T05:50:41.044865+0000","last_unstale":"2024-04-22T05:54:43.171572+0000","last_undegraded":"2024-04-22T05:54:43.171572+0000","last_fullsized":"2024-04-22T05:54:43.171572+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:37:43.523824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"22'5431","reported_seq":5445,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:17.517971+0000","last_change":"2024-04-22T05:50:47.114255+0000","last_active":"2024-04-22T05:54:17.517971+0000","last_peered":"2024-04-22T05:54:17.517971+0000","last_clean":"2024-04-22T05:54:17.517971+0000","last_became_active":"2024-04-22T05:50:47.113405+0000","last_became_peered":"2024-04-22T05:50:47.113405+0000","last_unstale":"2024-04-22T05:54:17.517971+0000","last_undegraded":"2024-04-22T05:54:17.517971+0000","last_fullsized":"2024-04-22T05:54:17.517971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:16.829289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128647688,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5434,"num_write_kb":203340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.171887+0000","last_change":"2024-04-22T05:50:41.052651+0000","last_active":"2024-04-22T05:54:02.171887+0000","last_peered":"2024-04-22T05:54:02.171887+0000","last_clean":"2024-04-22T05:54:02.171887+0000","last_became_active":"2024-04-22T05:50:41.052349+0000","last_became_peered":"2024-04-22T05:50:41.052349+0000","last_unstale":"2024-04-22T05:54:02.171887+0000","last_undegraded":"2024-04-22T05:54:02.171887+0000","last_fullsized":"2024-04-22T05:54:02.171887+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:46:51.494717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:43.171606+0000","last_change":"2024-04-22T05:50:40.047233+0000","last_active":"2024-04-22T05:54:43.171606+0000","last_peered":"2024-04-22T05:54:43.171606+0000","last_clean":"2024-04-22T05:54:43.171606+0000","last_became_active":"2024-04-22T05:50:40.046329+0000","last_became_peered":"2024-04-22T05:50:40.046329+0000","last_unstale":"2024-04-22T05:54:43.171606+0000","last_undegraded":"2024-04-22T05:54:43.171606+0000","last_fullsized":"2024-04-22T05:54:43.171606+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:39.011105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:39.011105+0000","last_clean_scrub_stamp":"2024-04-22T05:50:39.011105+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:16:02.587654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"22'3929","reported_seq":3945,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.591331+0000","last_change":"2024-04-22T05:50:47.116127+0000","last_active":"2024-04-22T05:54:02.591331+0000","last_peered":"2024-04-22T05:54:02.591331+0000","last_clean":"2024-04-22T05:54:02.591331+0000","last_became_active":"2024-04-22T05:50:47.115449+0000","last_became_peered":"2024-04-22T05:50:47.115449+0000","last_unstale":"2024-04-22T05:54:02.591331+0000","last_undegraded":"2024-04-22T05:54:02.591331+0000","last_fullsized":"2024-04-22T05:54:02.591331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:33.600047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":3867,"num_write_kb":173187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"22'10215","reported_seq":10334,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:55:07.352677+0000","last_change":"2024-04-22T05:50:47.173998+0000","last_active":"2024-04-22T05:55:07.352677+0000","last_peered":"2024-04-22T05:55:07.352677+0000","last_clean":"2024-04-22T05:55:07.352677+0000","last_became_active":"2024-04-22T05:50:47.173440+0000","last_became_peered":"2024-04-22T05:50:47.173440+0000","last_unstale":"2024-04-22T05:55:07.352677+0000","last_undegraded":"2024-04-22T05:55:07.352677+0000","last_fullsized":"2024-04-22T05:55:07.352677+0000","mapping_epoch":19,"log_start":"21'200","ondisk_log_start":"21'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:40.518919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":356,"num_read_kb":316,"num_write":8415,"num_write_kb":237085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"22'5035","reported_seq":5048,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:22.686699+0000","last_change":"2024-04-22T05:50:47.121542+0000","last_active":"2024-04-22T05:54:22.686699+0000","last_peered":"2024-04-22T05:54:22.686699+0000","last_clean":"2024-04-22T05:54:22.686699+0000","last_became_active":"2024-04-22T05:50:47.120541+0000","last_became_peered":"2024-04-22T05:50:47.120541+0000","last_unstale":"2024-04-22T05:54:22.686699+0000","last_undegraded":"2024-04-22T05:54:22.686699+0000","last_fullsized":"2024-04-22T05:54:22.686699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5035,"log_dups_size":0,"ondisk_log_size":5035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:07:12.244690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5009,"num_write_kb":193539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"21'4714","reported_seq":4730,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.160520+0000","last_change":"2024-04-22T05:50:47.123811+0000","last_active":"2024-04-22T05:54:02.160520+0000","last_peered":"2024-04-22T05:54:02.160520+0000","last_clean":"2024-04-22T05:54:02.160520+0000","last_became_active":"2024-04-22T05:50:47.123115+0000","last_became_peered":"2024-04-22T05:50:47.123115+0000","last_unstale":"2024-04-22T05:54:02.160520+0000","last_undegraded":"2024-04-22T05:54:02.160520+0000","last_fullsized":"2024-04-22T05:54:02.160520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:05:32.458383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600592,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":4720,"num_write_kb":218508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"22'4376","reported_seq":4392,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.593981+0000","last_change":"2024-04-22T05:50:47.116574+0000","last_active":"2024-04-22T05:54:02.593981+0000","last_peered":"2024-04-22T05:54:02.593981+0000","last_clean":"2024-04-22T05:54:02.593981+0000","last_became_active":"2024-04-22T05:50:47.114834+0000","last_became_peered":"2024-04-22T05:50:47.114834+0000","last_unstale":"2024-04-22T05:54:02.593981+0000","last_undegraded":"2024-04-22T05:54:02.593981+0000","last_fullsized":"2024-04-22T05:54:02.593981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4376,"log_dups_size":0,"ondisk_log_size":4376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:37:17.012651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":4338,"num_write_kb":218182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'3554","reported_seq":3567,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.101329+0000","last_change":"2024-04-22T05:50:47.117704+0000","last_active":"2024-04-22T05:55:18.101329+0000","last_peered":"2024-04-22T05:55:18.101329+0000","last_clean":"2024-04-22T05:55:18.101329+0000","last_became_active":"2024-04-22T05:50:47.113260+0000","last_became_peered":"2024-04-22T05:50:47.113260+0000","last_unstale":"2024-04-22T05:55:18.101329+0000","last_undegraded":"2024-04-22T05:55:18.101329+0000","last_fullsized":"2024-04-22T05:55:18.101329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:54:55.343145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104202240,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3525,"num_write_kb":160132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'3630","reported_seq":3736,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.179885+0000","last_change":"2024-04-22T05:50:47.123901+0000","last_active":"2024-04-22T05:55:18.179885+0000","last_peered":"2024-04-22T05:55:18.179885+0000","last_clean":"2024-04-22T05:55:18.179885+0000","last_became_active":"2024-04-22T05:50:47.122822+0000","last_became_peered":"2024-04-22T05:50:47.122822+0000","last_unstale":"2024-04-22T05:55:18.179885+0000","last_undegraded":"2024-04-22T05:55:18.179885+0000","last_fullsized":"2024-04-22T05:55:18.179885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3630,"log_dups_size":0,"ondisk_log_size":3630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:05.611537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83894800,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":300,"num_read_kb":235,"num_write":3880,"num_write_kb":108782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'4331","reported_seq":4367,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.101140+0000","last_change":"2024-04-22T05:50:47.118515+0000","last_active":"2024-04-22T05:55:18.101140+0000","last_peered":"2024-04-22T05:55:18.101140+0000","last_clean":"2024-04-22T05:55:18.101140+0000","last_became_active":"2024-04-22T05:50:47.117734+0000","last_became_peered":"2024-04-22T05:50:47.117734+0000","last_unstale":"2024-04-22T05:55:18.101140+0000","last_undegraded":"2024-04-22T05:55:18.101140+0000","last_fullsized":"2024-04-22T05:55:18.101140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:33:43.659487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126091776,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":92,"num_read_kb":134,"num_write":4330,"num_write_kb":222405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"22'3195","reported_seq":3207,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.583389+0000","last_change":"2024-04-22T05:50:47.125698+0000","last_active":"2024-04-22T05:54:02.583389+0000","last_peered":"2024-04-22T05:54:02.583389+0000","last_clean":"2024-04-22T05:54:02.583389+0000","last_became_active":"2024-04-22T05:50:47.124711+0000","last_became_peered":"2024-04-22T05:50:47.124711+0000","last_unstale":"2024-04-22T05:54:02.583389+0000","last_undegraded":"2024-04-22T05:54:02.583389+0000","last_fullsized":"2024-04-22T05:54:02.583389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3195,"log_dups_size":0,"ondisk_log_size":3195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:06:16.139033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3149,"num_write_kb":120072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"21'3697","reported_seq":4016,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.080934+0000","last_change":"2024-04-22T05:50:47.116263+0000","last_active":"2024-04-22T05:54:02.080934+0000","last_peered":"2024-04-22T05:54:02.080934+0000","last_clean":"2024-04-22T05:54:02.080934+0000","last_became_active":"2024-04-22T05:50:47.115535+0000","last_became_peered":"2024-04-22T05:50:47.115535+0000","last_unstale":"2024-04-22T05:54:02.080934+0000","last_undegraded":"2024-04-22T05:54:02.080934+0000","last_fullsized":"2024-04-22T05:54:02.080934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:20.378220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92406272,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1026,"num_read_kb":919,"num_write":3912,"num_write_kb":173447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"22'3436","reported_seq":3451,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.588199+0000","last_change":"2024-04-22T05:50:47.129955+0000","last_active":"2024-04-22T05:54:02.588199+0000","last_peered":"2024-04-22T05:54:02.588199+0000","last_clean":"2024-04-22T05:54:02.588199+0000","last_became_active":"2024-04-22T05:50:47.129345+0000","last_became_peered":"2024-04-22T05:50:47.129345+0000","last_unstale":"2024-04-22T05:54:02.588199+0000","last_undegraded":"2024-04-22T05:54:02.588199+0000","last_fullsized":"2024-04-22T05:54:02.588199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3436,"log_dups_size":0,"ondisk_log_size":3436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:40.789469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":71,"num_write":3499,"num_write_kb":148421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"22'6848","reported_seq":6862,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:15.867722+0000","last_change":"2024-04-22T05:50:47.127371+0000","last_active":"2024-04-22T05:54:15.867722+0000","last_peered":"2024-04-22T05:54:15.867722+0000","last_clean":"2024-04-22T05:54:15.867722+0000","last_became_active":"2024-04-22T05:50:47.126563+0000","last_became_peered":"2024-04-22T05:50:47.126563+0000","last_unstale":"2024-04-22T05:54:15.867722+0000","last_undegraded":"2024-04-22T05:54:15.867722+0000","last_fullsized":"2024-04-22T05:54:15.867722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6848,"log_dups_size":0,"ondisk_log_size":6848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:16:53.013583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1,"num_read_kb":64,"num_write":6824,"num_write_kb":157279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3567514376,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":2535,"num_read_kb":2546,"num_write":157131,"num_write_kb":5626233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6796107776,"data_stored":6833678488,"data_compressed":194275682,"data_compressed_allocated":202047488,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":158503,"ondisk_log_size":158503,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11158,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":14,"seq":60129542201,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591948,"kb_used_data":408260,"kb_used_omap":1,"kb_used_meta":183678,"kb_avail":93779892,"statfs":{"total":96636764160,"available":96030609408,"internally_reserved":0,"allocated":418058240,"data_stored":419207031,"data_compressed":12754442,"data_compressed_allocated":13279232,"data_compressed_original":14893056,"omap_allocated":1594,"internal_metadata":188086726},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.84599999999999997,"15min":0.78800000000000003},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.95799999999999996,"5min":20.890000000000001,"15min":20.890000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.622,"15min":0.66000000000000003},"min":{"1min":0.22600000000000001,"5min":0.185,"15min":0.185},"max":{"1min":1.032,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.76100000000000001}]},{"osd":1,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66400000000000003,"15min":0.67200000000000004},"min":{"1min":0.24199999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.221,"15min":0.221},"max":{"1min":1.04,"5min":1.218,"15min":1.218},"last":0.83099999999999996}]},{"osd":3,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.747,"15min":0.86499999999999999},"min":{"1min":0.26100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.92700000000000005,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.82099999999999995,"15min":0.88700000000000001},"min":{"1min":0.33000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":1.111,"5min":3.27,"15min":3.27},"last":0.91600000000000004}]},{"osd":4,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81599999999999995,"15min":0.80200000000000005},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.879},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.86299999999999999,"15min":0.85499999999999998},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.27,"5min":1.54,"15min":1.54},"last":1.1000000000000001}]},{"osd":5,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82999999999999996,"15min":0.84599999999999997},"min":{"1min":0.39100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.389,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.36699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.6439999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.94099999999999995}]},{"osd":6,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.92800000000000005,"15min":0.95399999999999996},"min":{"1min":0.41699999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.204,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94499999999999995,"15min":0.97399999999999998},"min":{"1min":0.39400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.597,"5min":1.651,"15min":1.651},"last":1.0660000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.2869999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.879,"5min":1.0920000000000001,"15min":1.02},"min":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4750000000000001,"5min":17.995999999999999,"15min":17.995999999999999},"last":1.024}]}]},{"osd":6,"up_from":14,"seq":60129542204,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418576,"kb_used_data":311176,"kb_used_omap":1,"kb_used_meta":107390,"kb_avail":93953264,"statfs":{"total":96636764160,"available":96208142336,"internally_reserved":0,"allocated":318644224,"data_stored":318792523,"data_compressed":4811634,"data_compressed_allocated":5001216,"data_compressed_original":5484544,"omap_allocated":1594,"internal_metadata":109967814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":1.506,"15min":1.8380000000000001},"min":{"1min":0.373,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.0720000000000001,"5min":34.100000000000001,"15min":34.100000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":1.0589999999999999,"15min":0.85199999999999998},"min":{"1min":0.439,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.131,"5min":30.306000000000001,"15min":30.306000000000001},"last":1.026}]},{"osd":1,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.79200000000000004,"15min":0.73799999999999999},"min":{"1min":0.47799999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.2849999999999999,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.107},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.81599999999999995,"15min":0.77600000000000002},"min":{"1min":0.32600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":1.2430000000000001,"5min":1.74,"15min":1.74},"last":1.3089999999999999}]},{"osd":2,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.88500000000000001,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.216,"15min":0.216},"max":{"1min":1.3009999999999999,"5min":2.488,"15min":2.488},"last":1.212},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.92700000000000005,"15min":0.88700000000000001},"min":{"1min":0.503,"5min":0.31,"15min":0.31},"max":{"1min":1.2370000000000001,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":1.1850000000000001}]},{"osd":3,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84799999999999998,"15min":0.86099999999999999},"min":{"1min":0.56499999999999995,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.101,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.88},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0209999999999999,"15min":0.95999999999999996},"min":{"1min":0.45200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.149,"5min":13.539,"15min":13.539},"last":1.056}]},{"osd":4,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.89600000000000002,"15min":0.92900000000000005},"min":{"1min":0.60199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.242,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88500000000000001,"15min":0.86699999999999999},"min":{"1min":0.501,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.204,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.139}]},{"osd":5,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.101,"15min":1.2310000000000001},"min":{"1min":0.66700000000000004,"5min":0.312,"15min":0.312},"max":{"1min":1.1970000000000001,"5min":11.922000000000001,"15min":11.922000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1,"15min":0.995},"min":{"1min":0.46999999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3080000000000001,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.71899999999999997}]},{"osd":7,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0269999999999999,"15min":0.995},"min":{"1min":0.60599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.353,"5min":2.129,"15min":2.129},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.101,"15min":1.101},"min":{"1min":0.67300000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3879999999999999,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.96399999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542202,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1090848,"kb_used_data":843224,"kb_used_omap":1,"kb_used_meta":247614,"kb_avail":93280992,"statfs":{"total":96636764160,"available":95519735808,"internally_reserved":0,"allocated":863461376,"data_stored":869115087,"data_compressed":21037680,"data_compressed_allocated":21954560,"data_compressed_original":28217344,"omap_allocated":1594,"internal_metadata":253557190},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76400000000000001,"15min":0.68999999999999995},"min":{"1min":0.443,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.3919999999999999,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.875,"15min":0.91000000000000003},"min":{"1min":0.29299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.3300000000000001,"5min":4.6390000000000002,"15min":4.6390000000000002},"last":0.69199999999999995}]},{"osd":1,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.78900000000000003,"15min":0.71399999999999997},"min":{"1min":0.35799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":1.3500000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.129},{"interface":"front","average":{"1min":0.74199999999999999,"5min":1.0049999999999999,"15min":1.181},"min":{"1min":0.376,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.0449999999999999,"5min":8.202,"15min":8.202},"last":0.72199999999999998}]},{"osd":2,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.83299999999999996,"15min":0.78800000000000003},"min":{"1min":0.39900000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4299999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.90300000000000002,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.474,"5min":1.673,"15min":1.673},"last":1.1850000000000001}]},{"osd":3,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":2.0110000000000001,"15min":2.8210000000000002},"min":{"1min":0.33600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.423,"5min":44.186999999999998,"15min":44.186999999999998},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.081,"15min":1.198},"min":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1419999999999999,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.99199999999999999}]},{"osd":4,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.1659999999999999,"15min":1.3240000000000001},"min":{"1min":0.29899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.3520000000000001,"5min":11.438000000000001,"15min":11.438000000000001},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.98599999999999999,"15min":0.95999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.171,"5min":2.181,"15min":2.181},"last":0.84499999999999997}]},{"osd":6,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":0.93500000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1830000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":1.097,"15min":1.335},"min":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.222,"5min":11.474,"15min":11.474},"last":1.0129999999999999}]},{"osd":7,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.107,"15min":1.081},"min":{"1min":0.38200000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.542,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.01,"5min":1.1100000000000001,"15min":1.1579999999999999},"min":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3009999999999999,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.038}]}]},{"osd":4,"up_from":14,"seq":60129542203,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1920752,"kb_used_data":1416424,"kb_used_omap":1,"kb_used_meta":504318,"kb_avail":92451088,"statfs":{"total":96636764160,"available":94669914112,"internally_reserved":0,"allocated":1450418176,"data_stored":1463130155,"data_compressed":51243088,"data_compressed_allocated":53432320,"data_compressed_original":67096576,"omap_allocated":1594,"internal_metadata":516422086},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":0.91800000000000004,"15min":0.75800000000000001},"min":{"1min":0.28499999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":12.618,"5min":12.618,"15min":12.618},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.0409999999999999,"15min":0.92000000000000004},"min":{"1min":0.35699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":12.548,"5min":12.548,"15min":12.548},"last":0.78600000000000003}]},{"osd":1,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.751,"15min":0.755},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.1970000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.5509999999999999,"5min":0.96099999999999997,"15min":0.81799999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":12.449999999999999,"5min":12.449999999999999,"15min":12.449999999999999},"last":1.1910000000000001}]},{"osd":2,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.33900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.097,"5min":2.226,"15min":2.226},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.048,"15min":0.92800000000000005},"min":{"1min":0.42899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":12.673,"5min":12.673,"15min":12.673},"last":1.0569999999999999}]},{"osd":3,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.90400000000000003,"15min":0.88400000000000001},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.2070000000000001,"5min":3.5329999999999999,"15min":3.5329999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.41899999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.335,"5min":2.254,"15min":2.254},"last":1.012}]},{"osd":5,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.1040000000000001,"15min":1.0349999999999999},"min":{"1min":0.40200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":12.721,"5min":12.721,"15min":12.721},"last":1.25},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.95999999999999996},"min":{"1min":0.39500000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.369,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.96199999999999997}]},{"osd":6,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.1919999999999999,"15min":1.04},"min":{"1min":0.55600000000000005,"5min":0.312,"15min":0.312},"max":{"1min":12.765000000000001,"5min":12.765000000000001,"15min":12.765000000000001},"last":1.117},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0860000000000001,"15min":1.087},"min":{"1min":0.57699999999999996,"5min":0.372,"15min":0.372},"max":{"1min":1.431,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.302}]},{"osd":7,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.038,"15min":1.0569999999999999},"min":{"1min":0.51800000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.456,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.091,"15min":1.0629999999999999},"min":{"1min":0.505,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.6120000000000001,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.278}]}]},{"osd":0,"up_from":14,"seq":60129542203,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1378064,"kb_used_data":1017992,"kb_used_omap":1,"kb_used_meta":360062,"kb_avail":92993776,"statfs":{"total":96636764160,"available":95225626624,"internally_reserved":0,"allocated":1042423808,"data_stored":1047499431,"data_compressed":29839922,"data_compressed_allocated":30982144,"data_compressed_original":36835328,"omap_allocated":1593,"internal_metadata":368703943},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60599999999999998,"15min":0.55000000000000004},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.792,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.74199999999999999,"15min":0.80600000000000005},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1.4770000000000001,"5min":4.7489999999999997,"15min":4.7489999999999997},"last":0.76600000000000001}]},{"osd":2,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.68700000000000006,"15min":0.625},"min":{"1min":0.28399999999999997,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.90100000000000002},"min":{"1min":0.33200000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.6579999999999999,"5min":4.7880000000000003,"15min":4.7880000000000003},"last":1.05}]},{"osd":3,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.5429999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.79400000000000004,"15min":0.76300000000000001},"min":{"1min":0.315,"5min":0.307,"15min":0.307},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.97799999999999998}]},{"osd":4,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.01,"15min":1.085},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.232,"5min":4.8440000000000003,"15min":4.8440000000000003},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.91400000000000003,"15min":0.90400000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.258,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.177}]},{"osd":5,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.94799999999999995,"15min":0.93200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.47,"5min":2.722,"15min":2.722},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.0289999999999999,"15min":1.107},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.474,"5min":4.8860000000000001,"15min":4.8860000000000001},"last":1.127}]},{"osd":6,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.96199999999999997,"15min":0.88900000000000001},"min":{"1min":0.40000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.5449999999999999,"5min":1.76,"15min":1.76},"last":1.3160000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.94199999999999995,"15min":0.89300000000000002},"min":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3560000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.3700000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2629999999999999,"15min":1.4019999999999999},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.4980000000000002,"5min":8.6839999999999993,"15min":8.6839999999999993},"last":1.224},{"interface":"front","average":{"1min":1.304,"5min":1.1970000000000001,"15min":1.2509999999999999},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.4649999999999999,"5min":4.4649999999999999,"15min":4.4649999999999999},"last":1.4770000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1329560,"kb_used_data":959696,"kb_used_omap":1,"kb_used_meta":369854,"kb_avail":93042280,"statfs":{"total":96636764160,"available":95275294720,"internally_reserved":0,"allocated":982728704,"data_stored":987917207,"data_compressed":23928438,"data_compressed_allocated":24997888,"data_compressed_original":30998528,"omap_allocated":1589,"internal_metadata":378730955},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":1.3640000000000001,"15min":1.0680000000000001},"min":{"1min":0.45700000000000002,"5min":0.224,"15min":0.224},"max":{"1min":1.1839999999999999,"5min":46.598999999999997,"15min":46.598999999999997},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":1.0309999999999999,"15min":1.2709999999999999},"min":{"1min":0.32700000000000001,"5min":0.216,"15min":0.216},"max":{"1min":1.141,"5min":15.144,"15min":15.144},"last":1.099}]},{"osd":1,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.80900000000000005,"15min":0.77500000000000002},"min":{"1min":0.35699999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.347,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.018,"5min":0.89700000000000002,"15min":0.82699999999999996},"min":{"1min":0.48599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3300000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.2310000000000001}]},{"osd":2,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.89500000000000002,"15min":0.871},"min":{"1min":0.33000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.262,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.94899999999999995,"15min":0.873},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4419999999999999,"5min":3.387,"15min":3.387},"last":1.006}]},{"osd":3,"last update":"Mon Apr 22 05:54:50 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.999,"15min":0.85899999999999999},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3520000000000001,"5min":7.1699999999999999,"15min":7.1699999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.89600000000000002,"15min":0.82799999999999996},"min":{"1min":0.45600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.399,"5min":3.6299999999999999,"15min":3.6299999999999999},"last":1.2729999999999999}]},{"osd":4,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.258,"15min":1.2909999999999999},"min":{"1min":0.34399999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.4650000000000001,"5min":9.6620000000000008,"15min":9.6620000000000008},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.94799999999999995,"15min":0.90200000000000002},"min":{"1min":0.40600000000000003,"5min":0.252,"15min":0.252},"max":{"1min":1.335,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.05}]},{"osd":5,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.95399999999999996,"15min":0.89900000000000002},"min":{"1min":0.29299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.54,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.208},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1659999999999999,"15min":0.997},"min":{"1min":0.39100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.72,"5min":7.2430000000000003,"15min":7.2430000000000003},"last":1.3759999999999999}]},{"osd":6,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.04,"15min":0.95599999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5649999999999999,"5min":3.5939999999999999,"15min":3.5939999999999999},"last":1.143},{"interface":"front","average":{"1min":1.208,"5min":1.2450000000000001,"15min":1.0649999999999999},"min":{"1min":0.53400000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.794,"5min":7.2960000000000003,"15min":7.2960000000000003},"last":1.5640000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1412644,"kb_used_data":1016540,"kb_used_omap":1,"kb_used_meta":396094,"kb_avail":92959196,"statfs":{"total":96636764160,"available":95190216704,"internally_reserved":0,"allocated":1040936960,"data_stored":1044385363,"data_compressed":24230251,"data_compressed_allocated":25128960,"data_compressed_original":29396992,"omap_allocated":1592,"internal_metadata":405600712},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.69199999999999995,"15min":0.82399999999999995},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":1.079,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.63},{"interface":"front","average":{"1min":0.66000000000000003,"5min":1.4970000000000001,"15min":2.2440000000000002},"min":{"1min":0.28799999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.129,"5min":22.721,"15min":22.721},"last":1.24}]},{"osd":1,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.80700000000000005,"15min":0.95099999999999996},"min":{"1min":0.23999999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.5840000000000001,"5min":9.6560000000000006,"15min":9.6560000000000006},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.3100000000000001,"5min":2.028,"15min":2.028},"last":0.69799999999999995}]},{"osd":2,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":1.3009999999999999,"15min":1.619},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.3029999999999999,"5min":22.763999999999999,"15min":22.763999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.94299999999999995,"15min":1.0920000000000001},"min":{"1min":0.313,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.6190000000000002,"5min":9.7739999999999991,"15min":9.7739999999999991},"last":0.75700000000000001}]},{"osd":4,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.7270000000000001,"15min":1.8879999999999999},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.657,"5min":27.384,"15min":27.384},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.571,"15min":1.931},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.508,"5min":22.824000000000002,"15min":22.824000000000002},"last":0.92100000000000004}]},{"osd":5,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.762,"15min":2.3220000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.4039999999999999,"5min":25.565999999999999,"15min":25.565999999999999},"last":1.139},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.98799999999999999,"15min":0.93999999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.6829999999999998,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":1.2010000000000001}]},{"osd":6,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.6419999999999999,"15min":1.9570000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.706,"5min":22.794,"15min":22.794},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.006,"5min":1.171,"15min":1.323},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6240000000000001,"5min":9.9559999999999995,"15min":9.9559999999999995},"last":1.3180000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":2.4089999999999998,"15min":3.1589999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.796,"5min":36.770000000000003,"15min":36.770000000000003},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.4039999999999999,"15min":1.4059999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.9100000000000001,"5min":16.239000000000001,"15min":16.239000000000001},"last":1.4019999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":989284,"kb_used_data":665372,"kb_used_omap":1,"kb_used_meta":323902,"kb_avail":93382556,"statfs":{"total":96636764160,"available":95623737344,"internally_reserved":0,"allocated":681340928,"data_stored":685333599,"data_compressed":26441385,"data_compressed_allocated":27336704,"data_compressed_original":31993856,"omap_allocated":1594,"internal_metadata":331676102},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.66600000000000004,"15min":0.70699999999999996},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.83099999999999996,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.26200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.1679999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.79600000000000004}]},{"osd":2,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.26100000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.014,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.68600000000000005,"15min":0.61799999999999999},"min":{"1min":0.34599999999999997,"5min":0.22,"15min":0.22},"max":{"1min":1.0900000000000001,"5min":3.871,"15min":3.871},"last":1.0189999999999999}]},{"osd":3,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.29699999999999999,"5min":0.252,"15min":0.252},"max":{"1min":1.1970000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.083},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.83499999999999996,"15min":0.85199999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.256,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.877}]},{"osd":4,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.0409999999999999,"15min":1.1180000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.242,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.044,"15min":1.044},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":2.0880000000000001,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.2370000000000001}]},{"osd":5,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.98299999999999998,"15min":0.94499999999999995},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.2650000000000001,"5min":3.8460000000000001,"15min":3.8460000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.98799999999999999,"15min":0.96999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.325,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.173}]},{"osd":6,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97699999999999998,"15min":0.92300000000000004},"min":{"1min":0.38400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3999999999999999,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.268},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.98799999999999999,"15min":0.95799999999999996},"min":{"1min":0.42699999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.222}]},{"osd":7,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.042,"15min":1.004},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.1800000000000002,"5min":3.8159999999999998,"15min":3.8159999999999998},"last":1.369},{"interface":"front","average":{"1min":1.091,"5min":1.0720000000000001,"15min":1.0369999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.615,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.3500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1042149376,"data_stored":1046890840,"data_compressed":29834343,"data_compressed_allocated":30949376,"data_compressed_original":36323328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":681115648,"data_stored":685252972,"data_compressed":26441385,"data_compressed_allocated":27336704,"data_compressed_original":31993856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417832960,"data_stored":419126404,"data_compressed":12754442,"data_compressed_allocated":13279232,"data_compressed_original":14893056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1040711680,"data_stored":1044304736,"data_compressed":24230251,"data_compressed_allocated":25128960,"data_compressed_original":29396992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1450192896,"data_stored":1463049528,"data_compressed":51243088,"data_compressed_allocated":53432320,"data_compressed_original":67096576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":863236096,"data_stored":869034460,"data_compressed":21037680,"data_compressed_allocated":21954560,"data_compressed_original":28217344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":318418944,"data_stored":318711896,"data_compressed":4811634,"data_compressed_allocated":5001216,"data_compressed_original":5484544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":982450176,"data_stored":987307652,"data_compressed":23922859,"data_compressed_allocated":24965120,"data_compressed_original":30486528,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T05:55:19.444 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-22T05:55:19.444 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T05:55:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c2bb94640 1 Processor -- start 2024-04-22T05:55:19.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c2bb94640 1 -- start start 2024-04-22T05:55:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c2bb94640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4c2406e1a0 0x7f4c2413b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c2bb94640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4c240a7920 0x7f4c2413e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c2bb94640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c24071d00 con 0x7f4c2413b760 2024-04-22T05:55:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c2bb94640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2406f8e0 con 0x7f4c2406e1a0 2024-04-22T05:55:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c2bb94640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2406e580 con 0x7f4c240a7920 2024-04-22T05:55:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c29930640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4c2406e1a0 0x7f4c2413b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c29930640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4c2406e1a0 0x7f4c2413b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33998/0 (socket says 172.21.15.40:33998) 2024-04-22T05:55:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c29930640 1 -- 172.21.15.40:0/1538030229 learned_addr learned my addr 172.21.15.40:0/1538030229 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c2912f640 1 --2- 172.21.15.40:0/1538030229 >> v2:172.21.15.62:3300/0 conn(0x7f4c240a7920 0x7f4c2413e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c29930640 1 -- 172.21.15.40:0/1538030229 >> v2:172.21.15.62:3300/0 conn(0x7f4c240a7920 msgr2=0x7f4c2413e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c29930640 1 --2- 172.21.15.40:0/1538030229 >> v2:172.21.15.62:3300/0 conn(0x7f4c240a7920 0x7f4c2413e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c29930640 1 -- 172.21.15.40:0/1538030229 >> v1:172.21.15.62:6789/0 conn(0x7f4c2413b760 legacy=0x7f4c2413e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.570+0000 7f4c29930640 1 -- 172.21.15.40:0/1538030229 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c240a8f60 con 0x7f4c2406e1a0 2024-04-22T05:55:19.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c29930640 1 --2- 172.21.15.40:0/1538030229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c2406e1a0 0x7f4c2413b220 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4c240a8bb0 tx=0x7f4c2001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781ef99e92b206c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:19.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c2892e640 1 -- 172.21.15.40:0/1538030229 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c200ab600 con 0x7f4c2406e1a0 2024-04-22T05:55:19.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c2892e640 1 -- 172.21.15.40:0/1538030229 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c200ab760 con 0x7f4c2406e1a0 2024-04-22T05:55:19.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c2892e640 1 -- 172.21.15.40:0/1538030229 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c200abaf0 con 0x7f4c2406e1a0 2024-04-22T05:55:19.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c2bb94640 1 -- 172.21.15.40:0/1538030229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c2406e1a0 msgr2=0x7f4c2413b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c2bb94640 1 --2- 172.21.15.40:0/1538030229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c2406e1a0 0x7f4c2413b220 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4c240a8bb0 tx=0x7f4c2001c3e0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c2bb94640 1 -- 172.21.15.40:0/1538030229 shutdown_connections 2024-04-22T05:55:19.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c2bb94640 1 --2- 172.21.15.40:0/1538030229 >> v2:172.21.15.62:3300/0 conn(0x7f4c240a7920 0x7f4c2413e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c2bb94640 1 --2- 172.21.15.40:0/1538030229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c2406e1a0 0x7f4c2413b220 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c2bb94640 1 -- 172.21.15.40:0/1538030229 >> 172.21.15.40:0/1538030229 conn(0x7f4c240b3170 msgr2=0x7f4c240b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:19.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c2bb94640 1 -- 172.21.15.40:0/1538030229 shutdown_connections 2024-04-22T05:55:19.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c2bb94640 1 -- 172.21.15.40:0/1538030229 wait complete. 2024-04-22T05:55:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c2bb94640 1 Processor -- start 2024-04-22T05:55:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.574+0000 7f4c2bb94640 1 -- start start 2024-04-22T05:55:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2bb94640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c2406e1a0 0x7f4c240aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2bb94640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c240a7920 0x7f4c240a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2bb94640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c2413b760 0x7f4c24130190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2bb94640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24134ba0 con 0x7f4c240a7920 2024-04-22T05:55:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2bb94640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24137790 con 0x7f4c2406e1a0 2024-04-22T05:55:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2bb94640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24136920 con 0x7f4c2413b760 2024-04-22T05:55:19.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2912f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c240a7920 0x7f4c240a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2912f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c240a7920 0x7f4c240a9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34010/0 (socket says 172.21.15.40:34010) 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2912f640 1 -- 172.21.15.40:0/2615710450 learned_addr learned my addr 172.21.15.40:0/2615710450 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2a131640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c2413b760 0x7f4c24130190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c29930640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c2406e1a0 0x7f4c240aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2912f640 1 -- 172.21.15.40:0/2615710450 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c2413b760 msgr2=0x7f4c24130190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2912f640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c2413b760 0x7f4c24130190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2912f640 1 -- 172.21.15.40:0/2615710450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c2406e1a0 msgr2=0x7f4c240aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2912f640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c2406e1a0 0x7f4c240aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2912f640 1 -- 172.21.15.40:0/2615710450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c200094d0 con 0x7f4c240a7920 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2912f640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c240a7920 0x7f4c240a9a60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800e900 tx=0x7f4c1800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c167fc640 1 -- 172.21.15.40:0/2615710450 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18009690 con 0x7f4c240a7920 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c167fc640 1 -- 172.21.15.40:0/2615710450 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1800ca00 con 0x7f4c240a7920 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2bb94640 1 -- 172.21.15.40:0/2615710450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c24137420 con 0x7f4c240a7920 2024-04-22T05:55:19.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c167fc640 1 -- 172.21.15.40:0/2615710450 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1800cdb0 con 0x7f4c240a7920 2024-04-22T05:55:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2bb94640 1 -- 172.21.15.40:0/2615710450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c241308a0 con 0x7f4c240a7920 2024-04-22T05:55:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c2bb94640 1 -- 172.21.15.40:0/2615710450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c24067380 con 0x7f4c240a7920 2024-04-22T05:55:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c167fc640 1 -- 172.21.15.40:0/2615710450 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4c18013470 con 0x7f4c240a7920 2024-04-22T05:55:19.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c167fc640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4c08066690 0x7f4c08068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c167fc640 1 -- 172.21.15.40:0/2615710450 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f4c180842b0 con 0x7f4c240a7920 2024-04-22T05:55:19.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c29930640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4c08066690 0x7f4c08068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.578+0000 7f4c29930640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4c08066690 0x7f4c08068b50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4c200af000 tx=0x7f4c200a7000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.582+0000 7f4c167fc640 1 -- 172.21.15.40:0/2615710450 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f4c18051780 con 0x7f4c240a7920 2024-04-22T05:55:19.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.734+0000 7f4c2bb94640 1 -- 172.21.15.40:0/2615710450 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c241375b0 con 0x7f4c08066690 2024-04-22T05:55:19.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.750+0000 7f4c167fc640 1 -- 172.21.15.40:0/2615710450 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+153354 (secure 0 0 0) 0x7f4c241375b0 con 0x7f4c08066690 2024-04-22T05:55:19.755 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:55:19.756 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-22T05:55:19.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.750+0000 7f4c2bb94640 1 -- 172.21.15.40:0/2615710450 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4c08066690 msgr2=0x7f4c08068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.750+0000 7f4c2bb94640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4c08066690 0x7f4c08068b50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4c200af000 tx=0x7f4c200a7000 comp rx=0 tx=0).stop 2024-04-22T05:55:19.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.750+0000 7f4c2bb94640 1 -- 172.21.15.40:0/2615710450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c240a7920 msgr2=0x7f4c240a9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.750+0000 7f4c2bb94640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c240a7920 0x7f4c240a9a60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800e900 tx=0x7f4c1800edc0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.750+0000 7f4c2bb94640 1 -- 172.21.15.40:0/2615710450 shutdown_connections 2024-04-22T05:55:19.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.750+0000 7f4c2bb94640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4c08066690 0x7f4c08068b50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.750+0000 7f4c2bb94640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f4c2413b760 0x7f4c24130190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.750+0000 7f4c2bb94640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c240a7920 0x7f4c240a9a60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.750+0000 7f4c2bb94640 1 --2- 172.21.15.40:0/2615710450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c2406e1a0 0x7f4c240aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.750+0000 7f4c2bb94640 1 -- 172.21.15.40:0/2615710450 >> 172.21.15.40:0/2615710450 conn(0x7f4c240b3170 msgr2=0x7f4c240b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:19.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.750+0000 7f4c2bb94640 1 -- 172.21.15.40:0/2615710450 shutdown_connections 2024-04-22T05:55:19.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.750+0000 7f4c2bb94640 1 -- 172.21.15.40:0/2615710450 wait complete. 2024-04-22T05:55:19.770 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-04-22T05:55:19.024834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3568104744,"num_objects":880,"num_object_clones":0,"num_object_copies":1760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":2651,"num_read_kb":2646,"num_write":157290,"num_write_kb":5628649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158620,"ondisk_log_size":158620,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9131676,"kb_used_data":6638684,"kb_used_omap":12,"kb_used_meta":2492915,"kb_avail":745843044,"statfs":{"total":773094113280,"available":763743277056,"internally_reserved":0,"allocated":6798012416,"data_stored":6835380396,"data_compressed":194286840,"data_compressed_allocated":202113024,"data_compressed_original":244916224,"omap_allocated":12744,"internal_metadata":2552745528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32768,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":12,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003234"},"pg_stats":[{"pgid":"3.1f","version":"21'3322","reported_seq":3348,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.159313+0000","last_change":"2024-04-22T05:50:47.121259+0000","last_active":"2024-04-22T05:54:02.159313+0000","last_peered":"2024-04-22T05:54:02.159313+0000","last_clean":"2024-04-22T05:54:02.159313+0000","last_became_active":"2024-04-22T05:50:47.120501+0000","last_became_peered":"2024-04-22T05:50:47.120501+0000","last_unstale":"2024-04-22T05:54:02.159313+0000","last_undegraded":"2024-04-22T05:54:02.159313+0000","last_fullsized":"2024-04-22T05:54:02.159313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3322,"log_dups_size":0,"ondisk_log_size":3322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:03:25.965635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109323296,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":3347,"num_write_kb":168857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"22'5496","reported_seq":5517,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:09.741154+0000","last_change":"2024-04-22T05:50:47.118054+0000","last_active":"2024-04-22T05:54:09.741154+0000","last_peered":"2024-04-22T05:54:09.741154+0000","last_clean":"2024-04-22T05:54:09.741154+0000","last_became_active":"2024-04-22T05:50:47.117274+0000","last_became_peered":"2024-04-22T05:50:47.117274+0000","last_unstale":"2024-04-22T05:54:09.741154+0000","last_undegraded":"2024-04-22T05:54:09.741154+0000","last_fullsized":"2024-04-22T05:54:09.741154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5496,"log_dups_size":0,"ondisk_log_size":5496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:26.168980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125960704,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":28,"num_read_kb":84,"num_write":5488,"num_write_kb":185156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"22'3417","reported_seq":3440,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:15.141589+0000","last_change":"2024-04-22T05:50:47.122061+0000","last_active":"2024-04-22T05:54:15.141589+0000","last_peered":"2024-04-22T05:54:15.141589+0000","last_clean":"2024-04-22T05:54:15.141589+0000","last_became_active":"2024-04-22T05:50:47.120601+0000","last_became_peered":"2024-04-22T05:50:47.120601+0000","last_unstale":"2024-04-22T05:54:15.141589+0000","last_undegraded":"2024-04-22T05:54:15.141589+0000","last_fullsized":"2024-04-22T05:54:15.141589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3417,"log_dups_size":0,"ondisk_log_size":3417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:30:19.535589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72812200,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":3406,"num_write_kb":94339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2981","reported_seq":2993,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-22T05:53:19.072284+0000","last_change":"2024-04-22T05:50:47.117092+0000","last_active":"2024-04-22T05:53:19.072284+0000","last_peered":"2024-04-22T05:53:19.072284+0000","last_clean":"2024-04-22T05:53:19.072284+0000","last_became_active":"2024-04-22T05:50:47.115803+0000","last_became_peered":"2024-04-22T05:50:47.115803+0000","last_unstale":"2024-04-22T05:53:19.072284+0000","last_undegraded":"2024-04-22T05:53:19.072284+0000","last_fullsized":"2024-04-22T05:53:19.072284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":2981,"log_dups_size":0,"ondisk_log_size":2981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:40:33.403824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89522176,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2976,"num_write_kb":149441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"22'5220","reported_seq":5255,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.593346+0000","last_change":"2024-04-22T05:50:47.124002+0000","last_active":"2024-04-22T05:54:02.593346+0000","last_peered":"2024-04-22T05:54:02.593346+0000","last_clean":"2024-04-22T05:54:02.593346+0000","last_became_active":"2024-04-22T05:50:47.123073+0000","last_became_peered":"2024-04-22T05:50:47.123073+0000","last_unstale":"2024-04-22T05:54:02.593346+0000","last_undegraded":"2024-04-22T05:54:02.593346+0000","last_fullsized":"2024-04-22T05:54:02.593346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5220,"log_dups_size":0,"ondisk_log_size":5220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:02:52.999238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150994944,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":90,"num_read_kb":78,"num_write":5209,"num_write_kb":201924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"22'6502","reported_seq":6522,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:10.515152+0000","last_change":"2024-04-22T05:50:47.118816+0000","last_active":"2024-04-22T05:54:10.515152+0000","last_peered":"2024-04-22T05:54:10.515152+0000","last_clean":"2024-04-22T05:54:10.515152+0000","last_became_active":"2024-04-22T05:50:47.118038+0000","last_became_peered":"2024-04-22T05:50:47.118038+0000","last_unstale":"2024-04-22T05:54:10.515152+0000","last_undegraded":"2024-04-22T05:54:10.515152+0000","last_fullsized":"2024-04-22T05:54:10.515152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:16:18.439809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113311744,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":6483,"num_write_kb":139343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"22'4753","reported_seq":4765,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:20.636889+0000","last_change":"2024-04-22T05:50:47.114124+0000","last_active":"2024-04-22T05:54:20.636889+0000","last_peered":"2024-04-22T05:54:20.636889+0000","last_clean":"2024-04-22T05:54:20.636889+0000","last_became_active":"2024-04-22T05:50:47.113368+0000","last_became_peered":"2024-04-22T05:50:47.113368+0000","last_unstale":"2024-04-22T05:54:20.636889+0000","last_undegraded":"2024-04-22T05:54:20.636889+0000","last_fullsized":"2024-04-22T05:54:20.636889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4753,"log_dups_size":0,"ondisk_log_size":4753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:26:25.586314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112566272,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":123245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"22'8053","reported_seq":8072,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:21.044772+0000","last_change":"2024-04-22T05:50:47.114390+0000","last_active":"2024-04-22T05:54:21.044772+0000","last_peered":"2024-04-22T05:54:21.044772+0000","last_clean":"2024-04-22T05:54:21.044772+0000","last_became_active":"2024-04-22T05:50:47.113569+0000","last_became_peered":"2024-04-22T05:50:47.113569+0000","last_unstale":"2024-04-22T05:54:21.044772+0000","last_undegraded":"2024-04-22T05:54:21.044772+0000","last_fullsized":"2024-04-22T05:54:21.044772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":8053,"log_dups_size":0,"ondisk_log_size":8053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:18:39.555599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":8065,"num_write_kb":185808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"22'6472","reported_seq":6495,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:55:07.378031+0000","last_change":"2024-04-22T05:50:47.122110+0000","last_active":"2024-04-22T05:55:07.378031+0000","last_peered":"2024-04-22T05:55:07.378031+0000","last_clean":"2024-04-22T05:55:07.378031+0000","last_became_active":"2024-04-22T05:50:47.121332+0000","last_became_peered":"2024-04-22T05:50:47.121332+0000","last_unstale":"2024-04-22T05:55:07.378031+0000","last_undegraded":"2024-04-22T05:55:07.378031+0000","last_fullsized":"2024-04-22T05:55:07.378031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6472,"log_dups_size":0,"ondisk_log_size":6472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:41.522456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111009792,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":6495,"num_write_kb":211703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"22'3761","reported_seq":3775,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.474346+0000","last_change":"2024-04-22T05:50:47.120024+0000","last_active":"2024-04-22T05:54:02.474346+0000","last_peered":"2024-04-22T05:54:02.474346+0000","last_clean":"2024-04-22T05:54:02.474346+0000","last_became_active":"2024-04-22T05:50:47.119031+0000","last_became_peered":"2024-04-22T05:50:47.119031+0000","last_unstale":"2024-04-22T05:54:02.474346+0000","last_undegraded":"2024-04-22T05:54:02.474346+0000","last_fullsized":"2024-04-22T05:54:02.474346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3761,"log_dups_size":0,"ondisk_log_size":3761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:17:48.991277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":3747,"num_write_kb":173061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"22'7962","reported_seq":7975,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:19.971677+0000","last_change":"2024-04-22T05:50:47.119142+0000","last_active":"2024-04-22T05:54:19.971677+0000","last_peered":"2024-04-22T05:54:19.971677+0000","last_clean":"2024-04-22T05:54:19.971677+0000","last_became_active":"2024-04-22T05:50:47.118399+0000","last_became_peered":"2024-04-22T05:50:47.118399+0000","last_unstale":"2024-04-22T05:54:19.971677+0000","last_undegraded":"2024-04-22T05:54:19.971677+0000","last_fullsized":"2024-04-22T05:54:19.971677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":7962,"log_dups_size":0,"ondisk_log_size":7962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:42:14.831153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7958,"num_write_kb":234882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"22'7046","reported_seq":7087,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:13.904714+0000","last_change":"2024-04-22T05:50:47.131544+0000","last_active":"2024-04-22T05:54:13.904714+0000","last_peered":"2024-04-22T05:54:13.904714+0000","last_clean":"2024-04-22T05:54:13.904714+0000","last_became_active":"2024-04-22T05:50:47.130842+0000","last_became_peered":"2024-04-22T05:50:47.130842+0000","last_unstale":"2024-04-22T05:54:13.904714+0000","last_undegraded":"2024-04-22T05:54:13.904714+0000","last_fullsized":"2024-04-22T05:54:13.904714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:14:10.141463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149815296,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":118,"num_read_kb":94,"num_write":7055,"num_write_kb":232332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.083465+0000","last_change":"2024-04-22T05:50:41.050585+0000","last_active":"2024-04-22T05:54:02.083465+0000","last_peered":"2024-04-22T05:54:02.083465+0000","last_clean":"2024-04-22T05:54:02.083465+0000","last_became_active":"2024-04-22T05:50:41.049696+0000","last_became_peered":"2024-04-22T05:50:41.049696+0000","last_unstale":"2024-04-22T05:54:02.083465+0000","last_undegraded":"2024-04-22T05:54:02.083465+0000","last_fullsized":"2024-04-22T05:54:02.083465+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:00:56.754144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'4443","reported_seq":4476,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:17.298588+0000","last_change":"2024-04-22T05:50:47.114576+0000","last_active":"2024-04-22T05:54:17.298588+0000","last_peered":"2024-04-22T05:54:17.298588+0000","last_clean":"2024-04-22T05:54:17.298588+0000","last_became_active":"2024-04-22T05:50:47.113838+0000","last_became_peered":"2024-04-22T05:50:47.113838+0000","last_unstale":"2024-04-22T05:54:17.298588+0000","last_undegraded":"2024-04-22T05:54:17.298588+0000","last_fullsized":"2024-04-22T05:54:17.298588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4443,"log_dups_size":0,"ondisk_log_size":4443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:29:28.189329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120194896,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":4449,"num_write_kb":155490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.160033+0000","last_change":"2024-04-22T05:50:41.048959+0000","last_active":"2024-04-22T05:54:02.160033+0000","last_peered":"2024-04-22T05:54:02.160033+0000","last_clean":"2024-04-22T05:54:02.160033+0000","last_became_active":"2024-04-22T05:50:41.048628+0000","last_became_peered":"2024-04-22T05:50:41.048628+0000","last_unstale":"2024-04-22T05:54:02.160033+0000","last_undegraded":"2024-04-22T05:54:02.160033+0000","last_fullsized":"2024-04-22T05:54:02.160033+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:46:57.799120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'3252","reported_seq":3267,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.591342+0000","last_change":"2024-04-22T05:50:47.130110+0000","last_active":"2024-04-22T05:54:02.591342+0000","last_peered":"2024-04-22T05:54:02.591342+0000","last_clean":"2024-04-22T05:54:02.591342+0000","last_became_active":"2024-04-22T05:50:47.129322+0000","last_became_peered":"2024-04-22T05:50:47.129322+0000","last_unstale":"2024-04-22T05:54:02.591342+0000","last_undegraded":"2024-04-22T05:54:02.591342+0000","last_fullsized":"2024-04-22T05:54:02.591342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3252,"log_dups_size":0,"ondisk_log_size":3252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:10:00.803610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":3214,"num_write_kb":155139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:53.173287+0000","last_change":"2024-04-22T05:50:41.341624+0000","last_active":"2024-04-22T05:54:53.173287+0000","last_peered":"2024-04-22T05:54:53.173287+0000","last_clean":"2024-04-22T05:54:53.173287+0000","last_became_active":"2024-04-22T05:50:41.340643+0000","last_became_peered":"2024-04-22T05:50:41.340643+0000","last_unstale":"2024-04-22T05:54:53.173287+0000","last_undegraded":"2024-04-22T05:54:53.173287+0000","last_fullsized":"2024-04-22T05:54:53.173287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:47:12.897727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'6361","reported_seq":6380,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:22.636657+0000","last_change":"2024-04-22T05:50:47.172954+0000","last_active":"2024-04-22T05:54:22.636657+0000","last_peered":"2024-04-22T05:54:22.636657+0000","last_clean":"2024-04-22T05:54:22.636657+0000","last_became_active":"2024-04-22T05:50:47.172121+0000","last_became_peered":"2024-04-22T05:50:47.172121+0000","last_unstale":"2024-04-22T05:54:22.636657+0000","last_undegraded":"2024-04-22T05:54:22.636657+0000","last_fullsized":"2024-04-22T05:54:22.636657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6361,"log_dups_size":0,"ondisk_log_size":6361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:17:03.619291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":26,"num_read_kb":18,"num_write":6371,"num_write_kb":148611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.159809+0000","last_change":"2024-04-22T05:50:41.341548+0000","last_active":"2024-04-22T05:54:02.159809+0000","last_peered":"2024-04-22T05:54:02.159809+0000","last_clean":"2024-04-22T05:54:02.159809+0000","last_became_active":"2024-04-22T05:50:41.341254+0000","last_became_peered":"2024-04-22T05:50:41.341254+0000","last_unstale":"2024-04-22T05:54:02.159809+0000","last_undegraded":"2024-04-22T05:54:02.159809+0000","last_fullsized":"2024-04-22T05:54:02.159809+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:34:01.082016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"21'3305","reported_seq":3329,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.174237+0000","last_change":"2024-04-22T05:50:47.121899+0000","last_active":"2024-04-22T05:54:02.174237+0000","last_peered":"2024-04-22T05:54:02.174237+0000","last_clean":"2024-04-22T05:54:02.174237+0000","last_became_active":"2024-04-22T05:50:47.120072+0000","last_became_peered":"2024-04-22T05:50:47.120072+0000","last_unstale":"2024-04-22T05:54:02.174237+0000","last_undegraded":"2024-04-22T05:54:02.174237+0000","last_fullsized":"2024-04-22T05:54:02.174237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3305,"log_dups_size":0,"ondisk_log_size":3305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:05:35.227673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99223376,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":3325,"num_write_kb":171289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'3629","reported_seq":3651,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.096142+0000","last_change":"2024-04-22T05:50:47.125635+0000","last_active":"2024-04-22T05:54:02.096142+0000","last_peered":"2024-04-22T05:54:02.096142+0000","last_clean":"2024-04-22T05:54:02.096142+0000","last_became_active":"2024-04-22T05:50:47.124841+0000","last_became_peered":"2024-04-22T05:50:47.124841+0000","last_unstale":"2024-04-22T05:54:02.096142+0000","last_undegraded":"2024-04-22T05:54:02.096142+0000","last_fullsized":"2024-04-22T05:54:02.096142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3629,"log_dups_size":0,"ondisk_log_size":3629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:08:34.775098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":3649,"num_write_kb":178556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.171665+0000","last_change":"2024-04-22T05:50:41.051239+0000","last_active":"2024-04-22T05:54:02.171665+0000","last_peered":"2024-04-22T05:54:02.171665+0000","last_clean":"2024-04-22T05:54:02.171665+0000","last_became_active":"2024-04-22T05:50:41.050711+0000","last_became_peered":"2024-04-22T05:50:41.050711+0000","last_unstale":"2024-04-22T05:54:02.171665+0000","last_undegraded":"2024-04-22T05:54:02.171665+0000","last_fullsized":"2024-04-22T05:54:02.171665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:07:59.435066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"22'5388","reported_seq":5404,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.590950+0000","last_change":"2024-04-22T05:50:47.173005+0000","last_active":"2024-04-22T05:54:02.590950+0000","last_peered":"2024-04-22T05:54:02.590950+0000","last_clean":"2024-04-22T05:54:02.590950+0000","last_became_active":"2024-04-22T05:50:47.172218+0000","last_became_peered":"2024-04-22T05:50:47.172218+0000","last_unstale":"2024-04-22T05:54:02.590950+0000","last_undegraded":"2024-04-22T05:54:02.590950+0000","last_fullsized":"2024-04-22T05:54:02.590950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5388,"log_dups_size":0,"ondisk_log_size":5388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:49:38.626473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138609688,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":5358,"num_write_kb":177059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.080259+0000","last_change":"2024-04-22T05:50:41.051710+0000","last_active":"2024-04-22T05:54:02.080259+0000","last_peered":"2024-04-22T05:54:02.080259+0000","last_clean":"2024-04-22T05:54:02.080259+0000","last_became_active":"2024-04-22T05:50:41.051259+0000","last_became_peered":"2024-04-22T05:50:41.051259+0000","last_unstale":"2024-04-22T05:54:02.080259+0000","last_undegraded":"2024-04-22T05:54:02.080259+0000","last_fullsized":"2024-04-22T05:54:02.080259+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:41:46.406807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"22'4949","reported_seq":5063,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.592758+0000","last_change":"2024-04-22T05:50:47.120166+0000","last_active":"2024-04-22T05:54:02.592758+0000","last_peered":"2024-04-22T05:54:02.592758+0000","last_clean":"2024-04-22T05:54:02.592758+0000","last_became_active":"2024-04-22T05:50:47.119414+0000","last_became_peered":"2024-04-22T05:50:47.119414+0000","last_unstale":"2024-04-22T05:54:02.592758+0000","last_undegraded":"2024-04-22T05:54:02.592758+0000","last_fullsized":"2024-04-22T05:54:02.592758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:46:02.360307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124520800,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":4927,"num_write_kb":205619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:43.171572+0000","last_change":"2024-04-22T05:50:41.045192+0000","last_active":"2024-04-22T05:54:43.171572+0000","last_peered":"2024-04-22T05:54:43.171572+0000","last_clean":"2024-04-22T05:54:43.171572+0000","last_became_active":"2024-04-22T05:50:41.044865+0000","last_became_peered":"2024-04-22T05:50:41.044865+0000","last_unstale":"2024-04-22T05:54:43.171572+0000","last_undegraded":"2024-04-22T05:54:43.171572+0000","last_fullsized":"2024-04-22T05:54:43.171572+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:37:43.523824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"22'5431","reported_seq":5445,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:17.517971+0000","last_change":"2024-04-22T05:50:47.114255+0000","last_active":"2024-04-22T05:54:17.517971+0000","last_peered":"2024-04-22T05:54:17.517971+0000","last_clean":"2024-04-22T05:54:17.517971+0000","last_became_active":"2024-04-22T05:50:47.113405+0000","last_became_peered":"2024-04-22T05:50:47.113405+0000","last_unstale":"2024-04-22T05:54:17.517971+0000","last_undegraded":"2024-04-22T05:54:17.517971+0000","last_fullsized":"2024-04-22T05:54:17.517971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:16.829289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128647688,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5434,"num_write_kb":203340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.171887+0000","last_change":"2024-04-22T05:50:41.052651+0000","last_active":"2024-04-22T05:54:02.171887+0000","last_peered":"2024-04-22T05:54:02.171887+0000","last_clean":"2024-04-22T05:54:02.171887+0000","last_became_active":"2024-04-22T05:50:41.052349+0000","last_became_peered":"2024-04-22T05:50:41.052349+0000","last_unstale":"2024-04-22T05:54:02.171887+0000","last_undegraded":"2024-04-22T05:54:02.171887+0000","last_fullsized":"2024-04-22T05:54:02.171887+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:46:51.494717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:43.171606+0000","last_change":"2024-04-22T05:50:40.047233+0000","last_active":"2024-04-22T05:54:43.171606+0000","last_peered":"2024-04-22T05:54:43.171606+0000","last_clean":"2024-04-22T05:54:43.171606+0000","last_became_active":"2024-04-22T05:50:40.046329+0000","last_became_peered":"2024-04-22T05:50:40.046329+0000","last_unstale":"2024-04-22T05:54:43.171606+0000","last_undegraded":"2024-04-22T05:54:43.171606+0000","last_fullsized":"2024-04-22T05:54:43.171606+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:39.011105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:39.011105+0000","last_clean_scrub_stamp":"2024-04-22T05:50:39.011105+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:16:02.587654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"22'3929","reported_seq":3945,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.591331+0000","last_change":"2024-04-22T05:50:47.116127+0000","last_active":"2024-04-22T05:54:02.591331+0000","last_peered":"2024-04-22T05:54:02.591331+0000","last_clean":"2024-04-22T05:54:02.591331+0000","last_became_active":"2024-04-22T05:50:47.115449+0000","last_became_peered":"2024-04-22T05:50:47.115449+0000","last_unstale":"2024-04-22T05:54:02.591331+0000","last_undegraded":"2024-04-22T05:54:02.591331+0000","last_fullsized":"2024-04-22T05:54:02.591331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:33.600047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":3867,"num_write_kb":173187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"22'10215","reported_seq":10334,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:55:07.352677+0000","last_change":"2024-04-22T05:50:47.173998+0000","last_active":"2024-04-22T05:55:07.352677+0000","last_peered":"2024-04-22T05:55:07.352677+0000","last_clean":"2024-04-22T05:55:07.352677+0000","last_became_active":"2024-04-22T05:50:47.173440+0000","last_became_peered":"2024-04-22T05:50:47.173440+0000","last_unstale":"2024-04-22T05:55:07.352677+0000","last_undegraded":"2024-04-22T05:55:07.352677+0000","last_fullsized":"2024-04-22T05:55:07.352677+0000","mapping_epoch":19,"log_start":"21'200","ondisk_log_start":"21'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:40.518919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":356,"num_read_kb":316,"num_write":8415,"num_write_kb":237085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"22'5035","reported_seq":5048,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:22.686699+0000","last_change":"2024-04-22T05:50:47.121542+0000","last_active":"2024-04-22T05:54:22.686699+0000","last_peered":"2024-04-22T05:54:22.686699+0000","last_clean":"2024-04-22T05:54:22.686699+0000","last_became_active":"2024-04-22T05:50:47.120541+0000","last_became_peered":"2024-04-22T05:50:47.120541+0000","last_unstale":"2024-04-22T05:54:22.686699+0000","last_undegraded":"2024-04-22T05:54:22.686699+0000","last_fullsized":"2024-04-22T05:54:22.686699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5035,"log_dups_size":0,"ondisk_log_size":5035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:07:12.244690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5009,"num_write_kb":193539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"21'4714","reported_seq":4730,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.160520+0000","last_change":"2024-04-22T05:50:47.123811+0000","last_active":"2024-04-22T05:54:02.160520+0000","last_peered":"2024-04-22T05:54:02.160520+0000","last_clean":"2024-04-22T05:54:02.160520+0000","last_became_active":"2024-04-22T05:50:47.123115+0000","last_became_peered":"2024-04-22T05:50:47.123115+0000","last_unstale":"2024-04-22T05:54:02.160520+0000","last_undegraded":"2024-04-22T05:54:02.160520+0000","last_fullsized":"2024-04-22T05:54:02.160520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:05:32.458383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600592,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":4720,"num_write_kb":218508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"22'4376","reported_seq":4392,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.593981+0000","last_change":"2024-04-22T05:50:47.116574+0000","last_active":"2024-04-22T05:54:02.593981+0000","last_peered":"2024-04-22T05:54:02.593981+0000","last_clean":"2024-04-22T05:54:02.593981+0000","last_became_active":"2024-04-22T05:50:47.114834+0000","last_became_peered":"2024-04-22T05:50:47.114834+0000","last_unstale":"2024-04-22T05:54:02.593981+0000","last_undegraded":"2024-04-22T05:54:02.593981+0000","last_fullsized":"2024-04-22T05:54:02.593981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4376,"log_dups_size":0,"ondisk_log_size":4376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:37:17.012651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":4338,"num_write_kb":218182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'3554","reported_seq":3567,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.101329+0000","last_change":"2024-04-22T05:50:47.117704+0000","last_active":"2024-04-22T05:55:18.101329+0000","last_peered":"2024-04-22T05:55:18.101329+0000","last_clean":"2024-04-22T05:55:18.101329+0000","last_became_active":"2024-04-22T05:50:47.113260+0000","last_became_peered":"2024-04-22T05:50:47.113260+0000","last_unstale":"2024-04-22T05:55:18.101329+0000","last_undegraded":"2024-04-22T05:55:18.101329+0000","last_fullsized":"2024-04-22T05:55:18.101329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:54:55.343145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104202240,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3525,"num_write_kb":160132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'3630","reported_seq":3736,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.179885+0000","last_change":"2024-04-22T05:50:47.123901+0000","last_active":"2024-04-22T05:55:18.179885+0000","last_peered":"2024-04-22T05:55:18.179885+0000","last_clean":"2024-04-22T05:55:18.179885+0000","last_became_active":"2024-04-22T05:50:47.122822+0000","last_became_peered":"2024-04-22T05:50:47.122822+0000","last_unstale":"2024-04-22T05:55:18.179885+0000","last_undegraded":"2024-04-22T05:55:18.179885+0000","last_fullsized":"2024-04-22T05:55:18.179885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3630,"log_dups_size":0,"ondisk_log_size":3630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:05.611537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83894800,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":300,"num_read_kb":235,"num_write":3880,"num_write_kb":108782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'4331","reported_seq":4367,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.101140+0000","last_change":"2024-04-22T05:50:47.118515+0000","last_active":"2024-04-22T05:55:18.101140+0000","last_peered":"2024-04-22T05:55:18.101140+0000","last_clean":"2024-04-22T05:55:18.101140+0000","last_became_active":"2024-04-22T05:50:47.117734+0000","last_became_peered":"2024-04-22T05:50:47.117734+0000","last_unstale":"2024-04-22T05:55:18.101140+0000","last_undegraded":"2024-04-22T05:55:18.101140+0000","last_fullsized":"2024-04-22T05:55:18.101140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:33:43.659487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126091776,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":92,"num_read_kb":134,"num_write":4330,"num_write_kb":222405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"22'3195","reported_seq":3207,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.583389+0000","last_change":"2024-04-22T05:50:47.125698+0000","last_active":"2024-04-22T05:54:02.583389+0000","last_peered":"2024-04-22T05:54:02.583389+0000","last_clean":"2024-04-22T05:54:02.583389+0000","last_became_active":"2024-04-22T05:50:47.124711+0000","last_became_peered":"2024-04-22T05:50:47.124711+0000","last_unstale":"2024-04-22T05:54:02.583389+0000","last_undegraded":"2024-04-22T05:54:02.583389+0000","last_fullsized":"2024-04-22T05:54:02.583389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3195,"log_dups_size":0,"ondisk_log_size":3195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:06:16.139033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3149,"num_write_kb":120072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"21'3697","reported_seq":4016,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.080934+0000","last_change":"2024-04-22T05:50:47.116263+0000","last_active":"2024-04-22T05:54:02.080934+0000","last_peered":"2024-04-22T05:54:02.080934+0000","last_clean":"2024-04-22T05:54:02.080934+0000","last_became_active":"2024-04-22T05:50:47.115535+0000","last_became_peered":"2024-04-22T05:50:47.115535+0000","last_unstale":"2024-04-22T05:54:02.080934+0000","last_undegraded":"2024-04-22T05:54:02.080934+0000","last_fullsized":"2024-04-22T05:54:02.080934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:20.378220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92406272,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1026,"num_read_kb":919,"num_write":3912,"num_write_kb":173447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"22'3436","reported_seq":3451,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.588199+0000","last_change":"2024-04-22T05:50:47.129955+0000","last_active":"2024-04-22T05:54:02.588199+0000","last_peered":"2024-04-22T05:54:02.588199+0000","last_clean":"2024-04-22T05:54:02.588199+0000","last_became_active":"2024-04-22T05:50:47.129345+0000","last_became_peered":"2024-04-22T05:50:47.129345+0000","last_unstale":"2024-04-22T05:54:02.588199+0000","last_undegraded":"2024-04-22T05:54:02.588199+0000","last_fullsized":"2024-04-22T05:54:02.588199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3436,"log_dups_size":0,"ondisk_log_size":3436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:09:40.789469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":71,"num_write":3499,"num_write_kb":148421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"22'6848","reported_seq":6862,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:15.867722+0000","last_change":"2024-04-22T05:50:47.127371+0000","last_active":"2024-04-22T05:54:15.867722+0000","last_peered":"2024-04-22T05:54:15.867722+0000","last_clean":"2024-04-22T05:54:15.867722+0000","last_became_active":"2024-04-22T05:50:47.126563+0000","last_became_peered":"2024-04-22T05:50:47.126563+0000","last_unstale":"2024-04-22T05:54:15.867722+0000","last_undegraded":"2024-04-22T05:54:15.867722+0000","last_fullsized":"2024-04-22T05:54:15.867722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6848,"log_dups_size":0,"ondisk_log_size":6848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:16:53.013583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1,"num_read_kb":64,"num_write":6824,"num_write_kb":157279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3567514376,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":2535,"num_read_kb":2546,"num_write":157131,"num_write_kb":5626233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6796107776,"data_stored":6833678488,"data_compressed":194275682,"data_compressed_allocated":202047488,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":158503,"ondisk_log_size":158503,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11158,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":14,"seq":60129542201,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591948,"kb_used_data":408260,"kb_used_omap":1,"kb_used_meta":183678,"kb_avail":93779892,"statfs":{"total":96636764160,"available":96030609408,"internally_reserved":0,"allocated":418058240,"data_stored":419207031,"data_compressed":12754442,"data_compressed_allocated":13279232,"data_compressed_original":14893056,"omap_allocated":1594,"internal_metadata":188086726},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.84599999999999997,"15min":0.78800000000000003},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.95799999999999996,"5min":20.890000000000001,"15min":20.890000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.622,"15min":0.66000000000000003},"min":{"1min":0.22600000000000001,"5min":0.185,"15min":0.185},"max":{"1min":1.032,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.76100000000000001}]},{"osd":1,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66400000000000003,"15min":0.67200000000000004},"min":{"1min":0.24199999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.221,"15min":0.221},"max":{"1min":1.04,"5min":1.218,"15min":1.218},"last":0.83099999999999996}]},{"osd":3,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.747,"15min":0.86499999999999999},"min":{"1min":0.26100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.92700000000000005,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.82099999999999995,"15min":0.88700000000000001},"min":{"1min":0.33000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":1.111,"5min":3.27,"15min":3.27},"last":0.91600000000000004}]},{"osd":4,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81599999999999995,"15min":0.80200000000000005},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.879},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.86299999999999999,"15min":0.85499999999999998},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.27,"5min":1.54,"15min":1.54},"last":1.1000000000000001}]},{"osd":5,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82999999999999996,"15min":0.84599999999999997},"min":{"1min":0.39100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.389,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.36699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.6439999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.94099999999999995}]},{"osd":6,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.92800000000000005,"15min":0.95399999999999996},"min":{"1min":0.41699999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.204,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94499999999999995,"15min":0.97399999999999998},"min":{"1min":0.39400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.597,"5min":1.651,"15min":1.651},"last":1.0660000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.2869999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.879,"5min":1.0920000000000001,"15min":1.02},"min":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4750000000000001,"5min":17.995999999999999,"15min":17.995999999999999},"last":1.024}]}]},{"osd":6,"up_from":14,"seq":60129542204,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418576,"kb_used_data":311176,"kb_used_omap":1,"kb_used_meta":107390,"kb_avail":93953264,"statfs":{"total":96636764160,"available":96208142336,"internally_reserved":0,"allocated":318644224,"data_stored":318792523,"data_compressed":4811634,"data_compressed_allocated":5001216,"data_compressed_original":5484544,"omap_allocated":1594,"internal_metadata":109967814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":1.506,"15min":1.8380000000000001},"min":{"1min":0.373,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.0720000000000001,"5min":34.100000000000001,"15min":34.100000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":1.0589999999999999,"15min":0.85199999999999998},"min":{"1min":0.439,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.131,"5min":30.306000000000001,"15min":30.306000000000001},"last":1.026}]},{"osd":1,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.79200000000000004,"15min":0.73799999999999999},"min":{"1min":0.47799999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.2849999999999999,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.107},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.81599999999999995,"15min":0.77600000000000002},"min":{"1min":0.32600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":1.2430000000000001,"5min":1.74,"15min":1.74},"last":1.3089999999999999}]},{"osd":2,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.88500000000000001,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.216,"15min":0.216},"max":{"1min":1.3009999999999999,"5min":2.488,"15min":2.488},"last":1.212},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.92700000000000005,"15min":0.88700000000000001},"min":{"1min":0.503,"5min":0.31,"15min":0.31},"max":{"1min":1.2370000000000001,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":1.1850000000000001}]},{"osd":3,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84799999999999998,"15min":0.86099999999999999},"min":{"1min":0.56499999999999995,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.101,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.88},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0209999999999999,"15min":0.95999999999999996},"min":{"1min":0.45200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.149,"5min":13.539,"15min":13.539},"last":1.056}]},{"osd":4,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.89600000000000002,"15min":0.92900000000000005},"min":{"1min":0.60199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.242,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88500000000000001,"15min":0.86699999999999999},"min":{"1min":0.501,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.204,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.139}]},{"osd":5,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.101,"15min":1.2310000000000001},"min":{"1min":0.66700000000000004,"5min":0.312,"15min":0.312},"max":{"1min":1.1970000000000001,"5min":11.922000000000001,"15min":11.922000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1,"15min":0.995},"min":{"1min":0.46999999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3080000000000001,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.71899999999999997}]},{"osd":7,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0269999999999999,"15min":0.995},"min":{"1min":0.60599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.353,"5min":2.129,"15min":2.129},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.101,"15min":1.101},"min":{"1min":0.67300000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3879999999999999,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.96399999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542202,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1090848,"kb_used_data":843224,"kb_used_omap":1,"kb_used_meta":247614,"kb_avail":93280992,"statfs":{"total":96636764160,"available":95519735808,"internally_reserved":0,"allocated":863461376,"data_stored":869115087,"data_compressed":21037680,"data_compressed_allocated":21954560,"data_compressed_original":28217344,"omap_allocated":1594,"internal_metadata":253557190},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76400000000000001,"15min":0.68999999999999995},"min":{"1min":0.443,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.3919999999999999,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.875,"15min":0.91000000000000003},"min":{"1min":0.29299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.3300000000000001,"5min":4.6390000000000002,"15min":4.6390000000000002},"last":0.69199999999999995}]},{"osd":1,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.78900000000000003,"15min":0.71399999999999997},"min":{"1min":0.35799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":1.3500000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.129},{"interface":"front","average":{"1min":0.74199999999999999,"5min":1.0049999999999999,"15min":1.181},"min":{"1min":0.376,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.0449999999999999,"5min":8.202,"15min":8.202},"last":0.72199999999999998}]},{"osd":2,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.83299999999999996,"15min":0.78800000000000003},"min":{"1min":0.39900000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4299999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.90300000000000002,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.474,"5min":1.673,"15min":1.673},"last":1.1850000000000001}]},{"osd":3,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":2.0110000000000001,"15min":2.8210000000000002},"min":{"1min":0.33600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.423,"5min":44.186999999999998,"15min":44.186999999999998},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.081,"15min":1.198},"min":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1419999999999999,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.99199999999999999}]},{"osd":4,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.1659999999999999,"15min":1.3240000000000001},"min":{"1min":0.29899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.3520000000000001,"5min":11.438000000000001,"15min":11.438000000000001},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.98599999999999999,"15min":0.95999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.171,"5min":2.181,"15min":2.181},"last":0.84499999999999997}]},{"osd":6,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":0.93500000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1830000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":1.097,"15min":1.335},"min":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.222,"5min":11.474,"15min":11.474},"last":1.0129999999999999}]},{"osd":7,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.107,"15min":1.081},"min":{"1min":0.38200000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.542,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.01,"5min":1.1100000000000001,"15min":1.1579999999999999},"min":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3009999999999999,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.038}]}]},{"osd":4,"up_from":14,"seq":60129542203,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1920752,"kb_used_data":1416424,"kb_used_omap":1,"kb_used_meta":504318,"kb_avail":92451088,"statfs":{"total":96636764160,"available":94669914112,"internally_reserved":0,"allocated":1450418176,"data_stored":1463130155,"data_compressed":51243088,"data_compressed_allocated":53432320,"data_compressed_original":67096576,"omap_allocated":1594,"internal_metadata":516422086},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":0.91800000000000004,"15min":0.75800000000000001},"min":{"1min":0.28499999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":12.618,"5min":12.618,"15min":12.618},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.0409999999999999,"15min":0.92000000000000004},"min":{"1min":0.35699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":12.548,"5min":12.548,"15min":12.548},"last":0.78600000000000003}]},{"osd":1,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.751,"15min":0.755},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.1970000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.5509999999999999,"5min":0.96099999999999997,"15min":0.81799999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":12.449999999999999,"5min":12.449999999999999,"15min":12.449999999999999},"last":1.1910000000000001}]},{"osd":2,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.33900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.097,"5min":2.226,"15min":2.226},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.048,"15min":0.92800000000000005},"min":{"1min":0.42899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":12.673,"5min":12.673,"15min":12.673},"last":1.0569999999999999}]},{"osd":3,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.90400000000000003,"15min":0.88400000000000001},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.2070000000000001,"5min":3.5329999999999999,"15min":3.5329999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.41899999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.335,"5min":2.254,"15min":2.254},"last":1.012}]},{"osd":5,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.1040000000000001,"15min":1.0349999999999999},"min":{"1min":0.40200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":12.721,"5min":12.721,"15min":12.721},"last":1.25},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.95999999999999996},"min":{"1min":0.39500000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.369,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.96199999999999997}]},{"osd":6,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.1919999999999999,"15min":1.04},"min":{"1min":0.55600000000000005,"5min":0.312,"15min":0.312},"max":{"1min":12.765000000000001,"5min":12.765000000000001,"15min":12.765000000000001},"last":1.117},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0860000000000001,"15min":1.087},"min":{"1min":0.57699999999999996,"5min":0.372,"15min":0.372},"max":{"1min":1.431,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.302}]},{"osd":7,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.038,"15min":1.0569999999999999},"min":{"1min":0.51800000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.456,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.091,"15min":1.0629999999999999},"min":{"1min":0.505,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.6120000000000001,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.278}]}]},{"osd":0,"up_from":14,"seq":60129542203,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1378064,"kb_used_data":1017992,"kb_used_omap":1,"kb_used_meta":360062,"kb_avail":92993776,"statfs":{"total":96636764160,"available":95225626624,"internally_reserved":0,"allocated":1042423808,"data_stored":1047499431,"data_compressed":29839922,"data_compressed_allocated":30982144,"data_compressed_original":36835328,"omap_allocated":1593,"internal_metadata":368703943},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60599999999999998,"15min":0.55000000000000004},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.792,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.74199999999999999,"15min":0.80600000000000005},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1.4770000000000001,"5min":4.7489999999999997,"15min":4.7489999999999997},"last":0.76600000000000001}]},{"osd":2,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.68700000000000006,"15min":0.625},"min":{"1min":0.28399999999999997,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.90100000000000002},"min":{"1min":0.33200000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.6579999999999999,"5min":4.7880000000000003,"15min":4.7880000000000003},"last":1.05}]},{"osd":3,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.5429999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.79400000000000004,"15min":0.76300000000000001},"min":{"1min":0.315,"5min":0.307,"15min":0.307},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.97799999999999998}]},{"osd":4,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.01,"15min":1.085},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.232,"5min":4.8440000000000003,"15min":4.8440000000000003},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.91400000000000003,"15min":0.90400000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.258,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.177}]},{"osd":5,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.94799999999999995,"15min":0.93200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.47,"5min":2.722,"15min":2.722},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.0289999999999999,"15min":1.107},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.474,"5min":4.8860000000000001,"15min":4.8860000000000001},"last":1.127}]},{"osd":6,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.96199999999999997,"15min":0.88900000000000001},"min":{"1min":0.40000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.5449999999999999,"5min":1.76,"15min":1.76},"last":1.3160000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.94199999999999995,"15min":0.89300000000000002},"min":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3560000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.3700000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2629999999999999,"15min":1.4019999999999999},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.4980000000000002,"5min":8.6839999999999993,"15min":8.6839999999999993},"last":1.224},{"interface":"front","average":{"1min":1.304,"5min":1.1970000000000001,"15min":1.2509999999999999},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.4649999999999999,"5min":4.4649999999999999,"15min":4.4649999999999999},"last":1.4770000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1329560,"kb_used_data":959696,"kb_used_omap":1,"kb_used_meta":369854,"kb_avail":93042280,"statfs":{"total":96636764160,"available":95275294720,"internally_reserved":0,"allocated":982728704,"data_stored":987917207,"data_compressed":23928438,"data_compressed_allocated":24997888,"data_compressed_original":30998528,"omap_allocated":1589,"internal_metadata":378730955},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":1.3640000000000001,"15min":1.0680000000000001},"min":{"1min":0.45700000000000002,"5min":0.224,"15min":0.224},"max":{"1min":1.1839999999999999,"5min":46.598999999999997,"15min":46.598999999999997},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":1.0309999999999999,"15min":1.2709999999999999},"min":{"1min":0.32700000000000001,"5min":0.216,"15min":0.216},"max":{"1min":1.141,"5min":15.144,"15min":15.144},"last":1.099}]},{"osd":1,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.80900000000000005,"15min":0.77500000000000002},"min":{"1min":0.35699999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.347,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.018,"5min":0.89700000000000002,"15min":0.82699999999999996},"min":{"1min":0.48599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3300000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.2310000000000001}]},{"osd":2,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.89500000000000002,"15min":0.871},"min":{"1min":0.33000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.262,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.94899999999999995,"15min":0.873},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4419999999999999,"5min":3.387,"15min":3.387},"last":1.006}]},{"osd":3,"last update":"Mon Apr 22 05:54:50 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.999,"15min":0.85899999999999999},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3520000000000001,"5min":7.1699999999999999,"15min":7.1699999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.89600000000000002,"15min":0.82799999999999996},"min":{"1min":0.45600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.399,"5min":3.6299999999999999,"15min":3.6299999999999999},"last":1.2729999999999999}]},{"osd":4,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.258,"15min":1.2909999999999999},"min":{"1min":0.34399999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.4650000000000001,"5min":9.6620000000000008,"15min":9.6620000000000008},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.94799999999999995,"15min":0.90200000000000002},"min":{"1min":0.40600000000000003,"5min":0.252,"15min":0.252},"max":{"1min":1.335,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.05}]},{"osd":5,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.95399999999999996,"15min":0.89900000000000002},"min":{"1min":0.29299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.54,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.208},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1659999999999999,"15min":0.997},"min":{"1min":0.39100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.72,"5min":7.2430000000000003,"15min":7.2430000000000003},"last":1.3759999999999999}]},{"osd":6,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.04,"15min":0.95599999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5649999999999999,"5min":3.5939999999999999,"15min":3.5939999999999999},"last":1.143},{"interface":"front","average":{"1min":1.208,"5min":1.2450000000000001,"15min":1.0649999999999999},"min":{"1min":0.53400000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.794,"5min":7.2960000000000003,"15min":7.2960000000000003},"last":1.5640000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1412644,"kb_used_data":1016540,"kb_used_omap":1,"kb_used_meta":396094,"kb_avail":92959196,"statfs":{"total":96636764160,"available":95190216704,"internally_reserved":0,"allocated":1040936960,"data_stored":1044385363,"data_compressed":24230251,"data_compressed_allocated":25128960,"data_compressed_original":29396992,"omap_allocated":1592,"internal_metadata":405600712},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.69199999999999995,"15min":0.82399999999999995},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":1.079,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.63},{"interface":"front","average":{"1min":0.66000000000000003,"5min":1.4970000000000001,"15min":2.2440000000000002},"min":{"1min":0.28799999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.129,"5min":22.721,"15min":22.721},"last":1.24}]},{"osd":1,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.80700000000000005,"15min":0.95099999999999996},"min":{"1min":0.23999999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.5840000000000001,"5min":9.6560000000000006,"15min":9.6560000000000006},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.3100000000000001,"5min":2.028,"15min":2.028},"last":0.69799999999999995}]},{"osd":2,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":1.3009999999999999,"15min":1.619},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.3029999999999999,"5min":22.763999999999999,"15min":22.763999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.94299999999999995,"15min":1.0920000000000001},"min":{"1min":0.313,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.6190000000000002,"5min":9.7739999999999991,"15min":9.7739999999999991},"last":0.75700000000000001}]},{"osd":4,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.7270000000000001,"15min":1.8879999999999999},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.657,"5min":27.384,"15min":27.384},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.571,"15min":1.931},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.508,"5min":22.824000000000002,"15min":22.824000000000002},"last":0.92100000000000004}]},{"osd":5,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.762,"15min":2.3220000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.4039999999999999,"5min":25.565999999999999,"15min":25.565999999999999},"last":1.139},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.98799999999999999,"15min":0.93999999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.6829999999999998,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":1.2010000000000001}]},{"osd":6,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.6419999999999999,"15min":1.9570000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.706,"5min":22.794,"15min":22.794},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.006,"5min":1.171,"15min":1.323},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6240000000000001,"5min":9.9559999999999995,"15min":9.9559999999999995},"last":1.3180000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":2.4089999999999998,"15min":3.1589999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.796,"5min":36.770000000000003,"15min":36.770000000000003},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.4039999999999999,"15min":1.4059999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.9100000000000001,"5min":16.239000000000001,"15min":16.239000000000001},"last":1.4019999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":989284,"kb_used_data":665372,"kb_used_omap":1,"kb_used_meta":323902,"kb_avail":93382556,"statfs":{"total":96636764160,"available":95623737344,"internally_reserved":0,"allocated":681340928,"data_stored":685333599,"data_compressed":26441385,"data_compressed_allocated":27336704,"data_compressed_original":31993856,"omap_allocated":1594,"internal_metadata":331676102},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.66600000000000004,"15min":0.70699999999999996},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.83099999999999996,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.26200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.1679999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.79600000000000004}]},{"osd":2,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.26100000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.014,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.68600000000000005,"15min":0.61799999999999999},"min":{"1min":0.34599999999999997,"5min":0.22,"15min":0.22},"max":{"1min":1.0900000000000001,"5min":3.871,"15min":3.871},"last":1.0189999999999999}]},{"osd":3,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.29699999999999999,"5min":0.252,"15min":0.252},"max":{"1min":1.1970000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.083},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.83499999999999996,"15min":0.85199999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.256,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.877}]},{"osd":4,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.0409999999999999,"15min":1.1180000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.242,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.044,"15min":1.044},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":2.0880000000000001,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.2370000000000001}]},{"osd":5,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.98299999999999998,"15min":0.94499999999999995},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.2650000000000001,"5min":3.8460000000000001,"15min":3.8460000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.98799999999999999,"15min":0.96999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.325,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.173}]},{"osd":6,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97699999999999998,"15min":0.92300000000000004},"min":{"1min":0.38400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3999999999999999,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.268},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.98799999999999999,"15min":0.95799999999999996},"min":{"1min":0.42699999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.222}]},{"osd":7,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.042,"15min":1.004},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.1800000000000002,"5min":3.8159999999999998,"15min":3.8159999999999998},"last":1.369},{"interface":"front","average":{"1min":1.091,"5min":1.0720000000000001,"15min":1.0369999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.615,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.3500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1042149376,"data_stored":1046890840,"data_compressed":29834343,"data_compressed_allocated":30949376,"data_compressed_original":36323328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":681115648,"data_stored":685252972,"data_compressed":26441385,"data_compressed_allocated":27336704,"data_compressed_original":31993856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417832960,"data_stored":419126404,"data_compressed":12754442,"data_compressed_allocated":13279232,"data_compressed_original":14893056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1040711680,"data_stored":1044304736,"data_compressed":24230251,"data_compressed_allocated":25128960,"data_compressed_original":29396992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1450192896,"data_stored":1463049528,"data_compressed":51243088,"data_compressed_allocated":53432320,"data_compressed_original":67096576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":863236096,"data_stored":869034460,"data_compressed":21037680,"data_compressed_allocated":21954560,"data_compressed_original":28217344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":318418944,"data_stored":318711896,"data_compressed":4811634,"data_compressed_allocated":5001216,"data_compressed_original":5484544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":982450176,"data_stored":987307652,"data_compressed":23922859,"data_compressed_allocated":24965120,"data_compressed_original":30486528,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T05:55:19.773 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T05:55:19.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.890+0000 7f6698183640 1 Processor -- start 2024-04-22T05:55:19.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6698183640 1 -- start start 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6698183640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66900a8ad0 0x7f66901356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6698183640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f66900a7740 0x7f6690135c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6698183640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6690136140 con 0x7f66901427d0 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6698183640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66900a9d10 con 0x7f66900a7740 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6698183640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66900a8eb0 con 0x7f66900a8ad0 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6695f1f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66900a8ad0 0x7f66901356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6696720640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f66901427d0 0x7f6690142c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36778/0 (socket says 172.21.15.40:36778) 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6696720640 1 -- 172.21.15.40:0/2048594331 learned_addr learned my addr 172.21.15.40:0/2048594331 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f669571e640 1 --2- 172.21.15.40:0/2048594331 >> v2:172.21.15.62:3300/0 conn(0x7f66900a7740 0x7f6690135c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6694f1d640 1 -- 172.21.15.40:0/2048594331 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929135009 0 0) 0x7f6690136140 con 0x7f66901427d0 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6694f1d640 1 -- 172.21.15.40:0/2048594331 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6678003530 con 0x7f66901427d0 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6695f1f640 1 -- 172.21.15.40:0/2048594331 >> v2:172.21.15.62:3300/0 conn(0x7f66900a7740 msgr2=0x7f6690135c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6695f1f640 1 --2- 172.21.15.40:0/2048594331 >> v2:172.21.15.62:3300/0 conn(0x7f66900a7740 0x7f6690135c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6695f1f640 1 -- 172.21.15.40:0/2048594331 >> v1:172.21.15.40:6789/0 conn(0x7f66901427d0 legacy=0x7f6690142c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6695f1f640 1 -- 172.21.15.40:0/2048594331 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66900a9fb0 con 0x7f66900a8ad0 2024-04-22T05:55:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6695f1f640 1 --2- 172.21.15.40:0/2048594331 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66900a8ad0 0x7f66901356c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f66880097c0 tx=0x7f668801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad05a46166c2275a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6694f1d640 1 -- 172.21.15.40:0/2048594331 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880ad660 con 0x7f66900a8ad0 2024-04-22T05:55:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.894+0000 7f6694f1d640 1 -- 172.21.15.40:0/2048594331 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66880ad7c0 con 0x7f66900a8ad0 2024-04-22T05:55:19.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6694f1d640 1 -- 172.21.15.40:0/2048594331 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880adb50 con 0x7f66900a8ad0 2024-04-22T05:55:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 -- 172.21.15.40:0/2048594331 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66900a8ad0 msgr2=0x7f66901356c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 --2- 172.21.15.40:0/2048594331 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66900a8ad0 0x7f66901356c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f66880097c0 tx=0x7f668801c450 comp rx=0 tx=0).stop 2024-04-22T05:55:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 -- 172.21.15.40:0/2048594331 shutdown_connections 2024-04-22T05:55:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 --2- 172.21.15.40:0/2048594331 >> v2:172.21.15.62:3300/0 conn(0x7f66900a7740 0x7f6690135c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 --2- 172.21.15.40:0/2048594331 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66900a8ad0 0x7f66901356c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 -- 172.21.15.40:0/2048594331 >> 172.21.15.40:0/2048594331 conn(0x7f66900b4280 msgr2=0x7f66900b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:19.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 -- 172.21.15.40:0/2048594331 shutdown_connections 2024-04-22T05:55:19.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 -- 172.21.15.40:0/2048594331 wait complete. 2024-04-22T05:55:19.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 Processor -- start 2024-04-22T05:55:19.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 -- start start 2024-04-22T05:55:19.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66900a7740 0x7f6690141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66900a8ad0 0x7f66901311d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66901427d0 0x7f66900af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66900aed90 con 0x7f66900a8ad0 2024-04-22T05:55:19.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669012bf40 con 0x7f66901427d0 2024-04-22T05:55:19.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6690070200 con 0x7f66900a7740 2024-04-22T05:55:19.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6695f1f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66900a7740 0x7f6690141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6695f1f640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66900a7740 0x7f6690141b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57502/0 (socket says 172.21.15.40:57502) 2024-04-22T05:55:19.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6695f1f640 1 -- 172.21.15.40:0/2380480370 learned_addr learned my addr 172.21.15.40:0/2380480370 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:19.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f669571e640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66900a8ad0 0x7f66901311d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6696720640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66901427d0 0x7f66900af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6695f1f640 1 -- 172.21.15.40:0/2380480370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66901427d0 msgr2=0x7f66900af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6695f1f640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66901427d0 0x7f66900af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6695f1f640 1 -- 172.21.15.40:0/2380480370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66900a8ad0 msgr2=0x7f66901311d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:19.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6695f1f640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66900a8ad0 0x7f66901311d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:19.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6695f1f640 1 -- 172.21.15.40:0/2380480370 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6688009420 con 0x7f66900a7740 2024-04-22T05:55:19.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f669571e640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66900a8ad0 0x7f66901311d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:19.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6695f1f640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66900a7740 0x7f6690141b80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f66880b0040 tx=0x7f668801ac90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:19.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f668e7fc640 1 -- 172.21.15.40:0/2380480370 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880b9650 con 0x7f66900a7740 2024-04-22T05:55:19.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f668e7fc640 1 -- 172.21.15.40:0/2380480370 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66880b97d0 con 0x7f66900a7740 2024-04-22T05:55:19.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 -- 172.21.15.40:0/2380480370 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6690130d40 con 0x7f66900a7740 2024-04-22T05:55:19.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f668e7fc640 1 -- 172.21.15.40:0/2380480370 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880b9b80 con 0x7f66900a7740 2024-04-22T05:55:19.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.898+0000 7f6698183640 1 -- 172.21.15.40:0/2380480370 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6690131710 con 0x7f66900a7740 2024-04-22T05:55:19.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.902+0000 7f6698183640 1 -- 172.21.15.40:0/2380480370 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66900aeed0 con 0x7f66900a7740 2024-04-22T05:55:19.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.902+0000 7f668e7fc640 1 -- 172.21.15.40:0/2380480370 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f66880b9ce0 con 0x7f66900a7740 2024-04-22T05:55:19.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.902+0000 7f668e7fc640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f667c066690 0x7f667c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:19.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.902+0000 7f668e7fc640 1 -- 172.21.15.40:0/2380480370 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f668811f100 con 0x7f66900a7740 2024-04-22T05:55:19.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.902+0000 7f669571e640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f667c066690 0x7f667c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:19.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.902+0000 7f669571e640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f667c066690 0x7f667c068b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6690070b50 tx=0x7f6684008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:19.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:19.906+0000 7f668e7fc640 1 -- 172.21.15.40:0/2380480370 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f66880ec5c0 con 0x7f66900a7740 2024-04-22T05:55:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.062+0000 7f6698183640 1 -- 172.21.15.40:0/2380480370 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f669004e690 con 0x7f66900a7740 2024-04-22T05:55:20.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.066+0000 7f668e7fc640 1 -- 172.21.15.40:0/2380480370 <== mon.2 v2:172.21.15.40:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15982 (secure 0 0 0) 0x7f66880efbe0 con 0x7f66900a7740 2024-04-22T05:55:20.068 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:55:20.069 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"36e9f223-2858-4559-8f74-0be5d30b8b0d","created":"2024-04-22T05:50:28.068495+0000","modified":"2024-04-22T05:55:18.092513+0000","last_up_change":"2024-04-22T05:50:37.978413+0000","last_in_change":"2024-04-22T05:50:30.434344+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T05:50:39.010995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-22T05:50:39.240779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephsqlite","create_time":"2024-04-22T05:50:45.888306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1997ac14-110c-4d7e-8ac5-d0ac4f70af75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":24983},{"type":"v1","addr":"172.21.15.40:6818","nonce":24983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":24983},{"type":"v1","addr":"172.21.15.40:6822","nonce":24983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":24983},{"type":"v1","addr":"172.21.15.40:6830","nonce":24983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":24983},{"type":"v1","addr":"172.21.15.40:6826","nonce":24983}]},"public_addr":"172.21.15.40:6818/24983","cluster_addr":"172.21.15.40:6822/24983","heartbeat_back_addr":"172.21.15.40:6830/24983","heartbeat_front_addr":"172.21.15.40:6826/24983","state":["exists","up"]},{"osd":1,"uuid":"cb6b2995-f7ce-4a68-ac16-d1690cc2b9ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":24984},{"type":"v1","addr":"172.21.15.40:6801","nonce":24984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":24984},{"type":"v1","addr":"172.21.15.40:6803","nonce":24984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":24984},{"type":"v1","addr":"172.21.15.40:6807","nonce":24984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":24984},{"type":"v1","addr":"172.21.15.40:6805","nonce":24984}]},"public_addr":"172.21.15.40:6801/24984","cluster_addr":"172.21.15.40:6803/24984","heartbeat_back_addr":"172.21.15.40:6807/24984","heartbeat_front_addr":"172.21.15.40:6805/24984","state":["exists","up"]},{"osd":2,"uuid":"a5634ce0-a08d-403f-bb05-eadb21417851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":24986},{"type":"v1","addr":"172.21.15.40:6809","nonce":24986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":24986},{"type":"v1","addr":"172.21.15.40:6811","nonce":24986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":24986},{"type":"v1","addr":"172.21.15.40:6815","nonce":24986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":24986},{"type":"v1","addr":"172.21.15.40:6813","nonce":24986}]},"public_addr":"172.21.15.40:6809/24986","cluster_addr":"172.21.15.40:6811/24986","heartbeat_back_addr":"172.21.15.40:6815/24986","heartbeat_front_addr":"172.21.15.40:6813/24986","state":["exists","up"]},{"osd":3,"uuid":"bada948d-c18e-4bf2-8902-9f383fcd1b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":24985},{"type":"v1","addr":"172.21.15.40:6819","nonce":24985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":24985},{"type":"v1","addr":"172.21.15.40:6823","nonce":24985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":24985},{"type":"v1","addr":"172.21.15.40:6831","nonce":24985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":24985},{"type":"v1","addr":"172.21.15.40:6827","nonce":24985}]},"public_addr":"172.21.15.40:6819/24985","cluster_addr":"172.21.15.40:6823/24985","heartbeat_back_addr":"172.21.15.40:6831/24985","heartbeat_front_addr":"172.21.15.40:6827/24985","state":["exists","up"]},{"osd":4,"uuid":"4ad76cfa-9e85-421e-bd2e-61006dd694bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":25135},{"type":"v1","addr":"172.21.15.62:6817","nonce":25135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":25135},{"type":"v1","addr":"172.21.15.62:6819","nonce":25135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":25135},{"type":"v1","addr":"172.21.15.62:6823","nonce":25135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":25135},{"type":"v1","addr":"172.21.15.62:6821","nonce":25135}]},"public_addr":"172.21.15.62:6817/25135","cluster_addr":"172.21.15.62:6819/25135","heartbeat_back_addr":"172.21.15.62:6823/25135","heartbeat_front_addr":"172.21.15.62:6821/25135","state":["exists","up"]},{"osd":5,"uuid":"b3b6f5a2-d949-429c-bfc4-b6a7333d8184","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":25137},{"type":"v1","addr":"172.21.15.62:6825","nonce":25137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":25137},{"type":"v1","addr":"172.21.15.62:6827","nonce":25137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":25137},{"type":"v1","addr":"172.21.15.62:6831","nonce":25137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":25137},{"type":"v1","addr":"172.21.15.62:6829","nonce":25137}]},"public_addr":"172.21.15.62:6825/25137","cluster_addr":"172.21.15.62:6827/25137","heartbeat_back_addr":"172.21.15.62:6831/25137","heartbeat_front_addr":"172.21.15.62:6829/25137","state":["exists","up"]},{"osd":6,"uuid":"00c5caf5-710a-4162-93fa-917a4823ffaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":25136},{"type":"v1","addr":"172.21.15.62:6801","nonce":25136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":25136},{"type":"v1","addr":"172.21.15.62:6803","nonce":25136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":25136},{"type":"v1","addr":"172.21.15.62:6807","nonce":25136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":25136},{"type":"v1","addr":"172.21.15.62:6805","nonce":25136}]},"public_addr":"172.21.15.62:6801/25136","cluster_addr":"172.21.15.62:6803/25136","heartbeat_back_addr":"172.21.15.62:6807/25136","heartbeat_front_addr":"172.21.15.62:6805/25136","state":["exists","up"]},{"osd":7,"uuid":"d17323e1-9cec-4bcb-b603-7f9fda35b5f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":25138},{"type":"v1","addr":"172.21.15.62:6809","nonce":25138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":25138},{"type":"v1","addr":"172.21.15.62:6811","nonce":25138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":25138},{"type":"v1","addr":"172.21.15.62:6815","nonce":25138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":25138},{"type":"v1","addr":"172.21.15.62:6813","nonce":25138}]},"public_addr":"172.21.15.62:6809/25138","cluster_addr":"172.21.15.62:6811/25138","heartbeat_back_addr":"172.21.15.62:6815/25138","heartbeat_front_addr":"172.21.15.62:6813/25138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.967711+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.944884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.961822+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.419674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.997808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:36.052542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:35.948229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T05:50:36.683341+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/4285456151":"2024-04-22T06:55:17.409514+0000","172.21.15.40:0/3804677017":"2024-04-22T06:54:01.332148+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T05:55:20.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.066+0000 7f6698183640 1 -- 172.21.15.40:0/2380480370 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f667c066690 msgr2=0x7f667c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:20.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.066+0000 7f6698183640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f667c066690 0x7f667c068b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6690070b50 tx=0x7f6684008040 comp rx=0 tx=0).stop 2024-04-22T05:55:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.066+0000 7f6698183640 1 -- 172.21.15.40:0/2380480370 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66900a7740 msgr2=0x7f6690141b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.066+0000 7f6698183640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66900a7740 0x7f6690141b80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f66880b0040 tx=0x7f668801ac90 comp rx=0 tx=0).stop 2024-04-22T05:55:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.066+0000 7f6698183640 1 -- 172.21.15.40:0/2380480370 shutdown_connections 2024-04-22T05:55:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.066+0000 7f6698183640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f667c066690 0x7f667c068b50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.066+0000 7f6698183640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66901427d0 0x7f66900af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.066+0000 7f6698183640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66900a8ad0 0x7f66901311d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.066+0000 7f6698183640 1 --2- 172.21.15.40:0/2380480370 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f66900a7740 0x7f6690141b80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.066+0000 7f6698183640 1 -- 172.21.15.40:0/2380480370 >> 172.21.15.40:0/2380480370 conn(0x7f66900b4280 msgr2=0x7f66901334f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.066+0000 7f6698183640 1 -- 172.21.15.40:0/2380480370 shutdown_connections 2024-04-22T05:55:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:20.070+0000 7f6698183640 1 -- 172.21.15.40:0/2380480370 wait complete. 2024-04-22T05:55:21.086 INFO:tasks.ceph:Scrubbing osd.0 2024-04-22T05:55:21.087 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-22T05:55:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe298487640 1 Processor -- start 2024-04-22T05:55:21.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe298487640 1 -- start start 2024-04-22T05:55:21.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe298487640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe290070e50 0x7fe29013dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe298487640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe29013ccf0 0x7fe29013d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe298487640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe29006e7d0 0x7fe29006ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe298487640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe290071230 con 0x7fe290070e50 2024-04-22T05:55:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe298487640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29013d6a0 con 0x7fe29013ccf0 2024-04-22T05:55:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe298487640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29013e670 con 0x7fe29006e7d0 2024-04-22T05:55:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe296a24640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe29006e7d0 0x7fe29006ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe296223640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe290070e50 0x7fe29013dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe296a24640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe29006e7d0 0x7fe29006ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57508/0 (socket says 172.21.15.40:57508) 2024-04-22T05:55:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe296a24640 1 -- 172.21.15.40:0/3514524681 learned_addr learned my addr 172.21.15.40:0/3514524681 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe295a22640 1 --2- 172.21.15.40:0/3514524681 >> v2:172.21.15.62:3300/0 conn(0x7fe29013ccf0 0x7fe29013d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe296a24640 1 -- 172.21.15.40:0/3514524681 >> v2:172.21.15.62:3300/0 conn(0x7fe29013ccf0 msgr2=0x7fe29013d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe296a24640 1 --2- 172.21.15.40:0/3514524681 >> v2:172.21.15.62:3300/0 conn(0x7fe29013ccf0 0x7fe29013d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe296a24640 1 -- 172.21.15.40:0/3514524681 >> v2:172.21.15.40:3300/0 conn(0x7fe290070e50 msgr2=0x7fe29013dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:21.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe296a24640 1 --2- 172.21.15.40:0/3514524681 >> v2:172.21.15.40:3300/0 conn(0x7fe290070e50 0x7fe29013dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.226+0000 7fe296a24640 1 -- 172.21.15.40:0/3514524681 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900a76f0 con 0x7fe29006e7d0 2024-04-22T05:55:21.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe296a24640 1 --2- 172.21.15.40:0/3514524681 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe29006e7d0 0x7fe29006ec20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe2900afa70 tx=0x7fe288020cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a103b3e570277f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe295221640 1 -- 172.21.15.40:0/3514524681 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2880a9ac0 con 0x7fe29006e7d0 2024-04-22T05:55:21.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe295221640 1 -- 172.21.15.40:0/3514524681 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2880a9c20 con 0x7fe29006e7d0 2024-04-22T05:55:21.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe295221640 1 -- 172.21.15.40:0/3514524681 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2880b1440 con 0x7fe29006e7d0 2024-04-22T05:55:21.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe298487640 1 -- 172.21.15.40:0/3514524681 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe29006e7d0 msgr2=0x7fe29006ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe298487640 1 --2- 172.21.15.40:0/3514524681 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe29006e7d0 0x7fe29006ec20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe2900afa70 tx=0x7fe288020cb0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe298487640 1 -- 172.21.15.40:0/3514524681 shutdown_connections 2024-04-22T05:55:21.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe298487640 1 --2- 172.21.15.40:0/3514524681 >> v2:172.21.15.62:3300/0 conn(0x7fe29013ccf0 0x7fe29013d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe298487640 1 --2- 172.21.15.40:0/3514524681 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe29006e7d0 0x7fe29006ec20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe298487640 1 --2- 172.21.15.40:0/3514524681 >> v2:172.21.15.40:3300/0 conn(0x7fe290070e50 0x7fe29013dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe298487640 1 -- 172.21.15.40:0/3514524681 >> 172.21.15.40:0/3514524681 conn(0x7fe2900736e0 msgr2=0x7fe290073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:21.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe298487640 1 -- 172.21.15.40:0/3514524681 shutdown_connections 2024-04-22T05:55:21.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe298487640 1 -- 172.21.15.40:0/3514524681 wait complete. 2024-04-22T05:55:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe298487640 1 Processor -- start 2024-04-22T05:55:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.230+0000 7fe298487640 1 -- start start 2024-04-22T05:55:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe298487640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe29006e7d0 0x7fe290140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe298487640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe290070e50 0x7fe2901340d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe298487640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe29013ccf0 0x7fe290134b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe298487640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290132be0 con 0x7fe29006e7d0 2024-04-22T05:55:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe298487640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290140930 con 0x7fe290070e50 2024-04-22T05:55:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe298487640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe290131870 con 0x7fe29013ccf0 2024-04-22T05:55:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe296223640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe29006e7d0 0x7fe290140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe296a24640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe29013ccf0 0x7fe290134b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe296223640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe29006e7d0 0x7fe290140050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34036/0 (socket says 172.21.15.40:34036) 2024-04-22T05:55:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe296223640 1 -- 172.21.15.40:0/1990674574 learned_addr learned my addr 172.21.15.40:0/1990674574 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe295a22640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe290070e50 0x7fe2901340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe296a24640 1 -- 172.21.15.40:0/1990674574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe290070e50 msgr2=0x7fe2901340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe296a24640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe290070e50 0x7fe2901340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe296a24640 1 -- 172.21.15.40:0/1990674574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe29006e7d0 msgr2=0x7fe290140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe296a24640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe29006e7d0 0x7fe290140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe296a24640 1 -- 172.21.15.40:0/1990674574 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28800f050 con 0x7fe29013ccf0 2024-04-22T05:55:21.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe295a22640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe290070e50 0x7fe2901340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:55:21.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe296223640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe29006e7d0 0x7fe290140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:21.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe296a24640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe29013ccf0 0x7fe290134b00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe2880b0000 tx=0x7fe28801da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe286ffd640 1 -- 172.21.15.40:0/1990674574 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2880bd860 con 0x7fe29013ccf0 2024-04-22T05:55:21.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe286ffd640 1 -- 172.21.15.40:0/1990674574 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2880bd9c0 con 0x7fe29013ccf0 2024-04-22T05:55:21.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe286ffd640 1 -- 172.21.15.40:0/1990674574 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2880bdd50 con 0x7fe29013ccf0 2024-04-22T05:55:21.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe298487640 1 -- 172.21.15.40:0/1990674574 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe290135740 con 0x7fe29013ccf0 2024-04-22T05:55:21.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe298487640 1 -- 172.21.15.40:0/1990674574 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe290139dd0 con 0x7fe29013ccf0 2024-04-22T05:55:21.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe298487640 1 -- 172.21.15.40:0/1990674574 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe290132d50 con 0x7fe29013ccf0 2024-04-22T05:55:21.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe286ffd640 1 -- 172.21.15.40:0/1990674574 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe2880b1440 con 0x7fe29013ccf0 2024-04-22T05:55:21.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe286ffd640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe274066690 0x7fe274068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe286ffd640 1 -- 172.21.15.40:0/1990674574 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7fe288123970 con 0x7fe29013ccf0 2024-04-22T05:55:21.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe286ffd640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] conn(0x7fe27406d7f0 0x7fe27406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe286ffd640 1 -- 172.21.15.40:0/1990674574 --> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2740702c0 con 0x7fe27406d7f0 2024-04-22T05:55:21.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe286ffd640 1 -- 172.21.15.40:0/1990674574 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe288123dc0 con 0x7fe29013ccf0 2024-04-22T05:55:21.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe295a22640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] conn(0x7fe27406d7f0 0x7fe27406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.234+0000 7fe296223640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe274066690 0x7fe274068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.238+0000 7fe295a22640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] conn(0x7fe27406d7f0 0x7fe27406fc10 crc :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.238+0000 7fe296223640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe274066690 0x7fe274068b50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe28c011310 tx=0x7fe28c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.238+0000 7fe286ffd640 1 -- 172.21.15.40:0/1990674574 <== osd.0 v2:172.21.15.40:6816/24983 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fe2740702c0 con 0x7fe27406d7f0 2024-04-22T05:55:21.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.250+0000 7fe298487640 1 -- 172.21.15.40:0/1990674574 --> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe29004e9c0 con 0x7fe27406d7f0 2024-04-22T05:55:21.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.250+0000 7fe286ffd640 1 -- 172.21.15.40:0/1990674574 <== osd.0 v2:172.21.15.40:6816/24983 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe29004e9c0 con 0x7fe27406d7f0 2024-04-22T05:55:21.252 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-22T05:55:21.252 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T05:55:21.252 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-22T05:55:21.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.250+0000 7fe298487640 1 -- 172.21.15.40:0/1990674574 >> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] conn(0x7fe27406d7f0 msgr2=0x7fe27406fc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.250+0000 7fe298487640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.40:6816/24983,v1:172.21.15.40:6818/24983] conn(0x7fe27406d7f0 0x7fe27406fc10 crc :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.250+0000 7fe298487640 1 -- 172.21.15.40:0/1990674574 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe274066690 msgr2=0x7fe274068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.250+0000 7fe298487640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fe274066690 0x7fe274068b50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe28c011310 tx=0x7fe28c009190 comp rx=0 tx=0).stop 2024-04-22T05:55:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.250+0000 7fe298487640 1 -- 172.21.15.40:0/1990674574 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe29013ccf0 msgr2=0x7fe290134b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.250+0000 7fe298487640 1 --2- 172.21.15.40:0/1990674574 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fe29013ccf0 0x7fe290134b00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe2880b0000 tx=0x7fe28801da50 comp rx=0 tx=0).stop 2024-04-22T05:55:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.250+0000 7fe296a24640 1 -- 172.21.15.40:0/1990674574 reap_dead start 2024-04-22T05:55:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.250+0000 7fe298487640 1 -- 172.21.15.40:0/1990674574 shutdown_connections 2024-04-22T05:55:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.250+0000 7fe298487640 1 -- 172.21.15.40:0/1990674574 >> 172.21.15.40:0/1990674574 conn(0x7fe2900736e0 msgr2=0x7fe29012fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.250+0000 7fe298487640 1 -- 172.21.15.40:0/1990674574 shutdown_connections 2024-04-22T05:55:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.250+0000 7fe298487640 1 -- 172.21.15.40:0/1990674574 wait complete. 2024-04-22T05:55:21.268 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-22T05:55:21.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f63814a7640 1 Processor -- start 2024-04-22T05:55:21.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f63814a7640 1 -- start start 2024-04-22T05:55:21.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f63814a7640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f637c0a79a0 0x7f637c0a7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f63814a7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f637c13e730 0x7f637c13ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f63814a7640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637c13f8b0 con 0x7f637c06df40 2024-04-22T05:55:21.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f63814a7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637c0a8bb0 con 0x7f637c13e730 2024-04-22T05:55:21.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f63814a7640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637c0a8cf0 con 0x7f637c0a79a0 2024-04-22T05:55:21.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f637b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f637c13e730 0x7f637c13ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f637b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f637c13e730 0x7f637c13ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34050/0 (socket says 172.21.15.40:34050) 2024-04-22T05:55:21.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f637b7fe640 1 -- 172.21.15.40:0/1328937899 learned_addr learned my addr 172.21.15.40:0/1328937899 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:21.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f637affd640 1 --2- 172.21.15.40:0/1328937899 >> v2:172.21.15.62:3300/0 conn(0x7f637c0a79a0 0x7f637c0a7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f637b7fe640 1 -- 172.21.15.40:0/1328937899 >> v2:172.21.15.62:3300/0 conn(0x7f637c0a79a0 msgr2=0x7f637c0a7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:21.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f637b7fe640 1 --2- 172.21.15.40:0/1328937899 >> v2:172.21.15.62:3300/0 conn(0x7f637c0a79a0 0x7f637c0a7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f637b7fe640 1 -- 172.21.15.40:0/1328937899 >> v1:172.21.15.62:6789/0 conn(0x7f637c06df40 legacy=0x7f637c13fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:21.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.382+0000 7f637b7fe640 1 -- 172.21.15.40:0/1328937899 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c0a8e70 con 0x7f637c13e730 2024-04-22T05:55:21.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f637b7fe640 1 --2- 172.21.15.40:0/1328937899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f637c13e730 0x7f637c13ebe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f637c06def0 tx=0x7f637401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609060d04b48176b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f6379ffb640 1 -- 172.21.15.40:0/1328937899 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63740a6430 con 0x7f637c13e730 2024-04-22T05:55:21.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f6379ffb640 1 -- 172.21.15.40:0/1328937899 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63740a6590 con 0x7f637c13e730 2024-04-22T05:55:21.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f6379ffb640 1 -- 172.21.15.40:0/1328937899 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63740a6920 con 0x7f637c13e730 2024-04-22T05:55:21.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 -- 172.21.15.40:0/1328937899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f637c13e730 msgr2=0x7f637c13ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 --2- 172.21.15.40:0/1328937899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f637c13e730 0x7f637c13ebe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f637c06def0 tx=0x7f637401c870 comp rx=0 tx=0).stop 2024-04-22T05:55:21.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 -- 172.21.15.40:0/1328937899 shutdown_connections 2024-04-22T05:55:21.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 --2- 172.21.15.40:0/1328937899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f637c13e730 0x7f637c13ebe0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 --2- 172.21.15.40:0/1328937899 >> v2:172.21.15.62:3300/0 conn(0x7f637c0a79a0 0x7f637c0a7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 -- 172.21.15.40:0/1328937899 >> 172.21.15.40:0/1328937899 conn(0x7f637c0b3170 msgr2=0x7f637c0b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:21.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 -- 172.21.15.40:0/1328937899 shutdown_connections 2024-04-22T05:55:21.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 -- 172.21.15.40:0/1328937899 wait complete. 2024-04-22T05:55:21.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 Processor -- start 2024-04-22T05:55:21.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 -- start start 2024-04-22T05:55:21.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f637c06df40 0x7f637c137dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f637c0a79a0 0x7f637c1db380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f637c13e730 0x7f637c128f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c128df0 con 0x7f637c0a79a0 2024-04-22T05:55:21.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1daef0 con 0x7f637c06df40 2024-04-22T05:55:21.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f63814a7640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637c137090 con 0x7f637c13e730 2024-04-22T05:55:21.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f637a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f637c0a79a0 0x7f637c1db380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f637a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f637c0a79a0 0x7f637c1db380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34054/0 (socket says 172.21.15.40:34054) 2024-04-22T05:55:21.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f637a7fc640 1 -- 172.21.15.40:0/1707318880 learned_addr learned my addr 172.21.15.40:0/1707318880 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:21.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f637b7fe640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f637c13e730 0x7f637c128f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f637affd640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f637c06df40 0x7f637c137dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f637a7fc640 1 -- 172.21.15.40:0/1707318880 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f637c13e730 msgr2=0x7f637c128f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f637a7fc640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f637c13e730 0x7f637c128f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f637a7fc640 1 -- 172.21.15.40:0/1707318880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f637c06df40 msgr2=0x7f637c137dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.386+0000 7f637a7fc640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f637c06df40 0x7f637c137dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.390+0000 7f637a7fc640 1 -- 172.21.15.40:0/1707318880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6374009b30 con 0x7f637c0a79a0 2024-04-22T05:55:21.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.390+0000 7f637a7fc640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f637c0a79a0 0x7f637c1db380 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f637000c410 tx=0x7f637000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.390+0000 7f63637fe640 1 -- 172.21.15.40:0/1707318880 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370011790 con 0x7f637c0a79a0 2024-04-22T05:55:21.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.390+0000 7f63637fe640 1 -- 172.21.15.40:0/1707318880 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63700118f0 con 0x7f637c0a79a0 2024-04-22T05:55:21.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.390+0000 7f63814a7640 1 -- 172.21.15.40:0/1707318880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c1373d0 con 0x7f637c0a79a0 2024-04-22T05:55:21.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.390+0000 7f63637fe640 1 -- 172.21.15.40:0/1707318880 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370011ce0 con 0x7f637c0a79a0 2024-04-22T05:55:21.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.390+0000 7f63814a7640 1 -- 172.21.15.40:0/1707318880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c1db8c0 con 0x7f637c0a79a0 2024-04-22T05:55:21.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.390+0000 7f63814a7640 1 -- 172.21.15.40:0/1707318880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637c067380 con 0x7f637c0a79a0 2024-04-22T05:55:21.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.390+0000 7f63637fe640 1 -- 172.21.15.40:0/1707318880 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f637001c460 con 0x7f637c0a79a0 2024-04-22T05:55:21.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.390+0000 7f63637fe640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6368066690 0x7f6368068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.390+0000 7f63637fe640 1 -- 172.21.15.40:0/1707318880 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f6370083f60 con 0x7f637c0a79a0 2024-04-22T05:55:21.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.390+0000 7f637affd640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6368066690 0x7f6368068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.390+0000 7f637affd640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6368066690 0x7f6368068b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f636c00b9d0 tx=0x7f636c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.394+0000 7f63637fe640 1 -- 172.21.15.40:0/1707318880 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f63700513a0 con 0x7f637c0a79a0 2024-04-22T05:55:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.554+0000 7f63814a7640 1 -- 172.21.15.40:0/1707318880 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f637c06e590 con 0x7f6368066690 2024-04-22T05:55:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.554+0000 7f63637fe640 1 -- 172.21.15.40:0/1707318880 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f636c00b670 con 0x7f6368066690 2024-04-22T05:55:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 0 to deep-scrub 2024-04-22T05:55:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.558+0000 7f63814a7640 1 -- 172.21.15.40:0/1707318880 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6368066690 msgr2=0x7f6368068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.558+0000 7f63814a7640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6368066690 0x7f6368068b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f636c00b9d0 tx=0x7f636c008040 comp rx=0 tx=0).stop 2024-04-22T05:55:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.558+0000 7f63814a7640 1 -- 172.21.15.40:0/1707318880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f637c0a79a0 msgr2=0x7f637c1db380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.558+0000 7f63814a7640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f637c0a79a0 0x7f637c1db380 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f637000c410 tx=0x7f637000c8d0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.558+0000 7f63814a7640 1 -- 172.21.15.40:0/1707318880 shutdown_connections 2024-04-22T05:55:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.558+0000 7f63814a7640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6368066690 0x7f6368068b50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.558+0000 7f63814a7640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f637c13e730 0x7f637c128f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.558+0000 7f63814a7640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f637c0a79a0 0x7f637c1db380 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.558+0000 7f63814a7640 1 --2- 172.21.15.40:0/1707318880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f637c06df40 0x7f637c137dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.558+0000 7f63814a7640 1 -- 172.21.15.40:0/1707318880 >> 172.21.15.40:0/1707318880 conn(0x7f637c0b3170 msgr2=0x7f637c0afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.558+0000 7f63814a7640 1 -- 172.21.15.40:0/1707318880 shutdown_connections 2024-04-22T05:55:21.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.558+0000 7f63814a7640 1 -- 172.21.15.40:0/1707318880 wait complete. 2024-04-22T05:55:21.577 INFO:tasks.ceph:Scrubbing osd.1 2024-04-22T05:55:21.577 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-22T05:55:21.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f06a1bca640 1 Processor -- start 2024-04-22T05:55:21.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f06a1bca640 1 -- start start 2024-04-22T05:55:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f06a1bca640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f069c070780 0x7f069c141c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f06a1bca640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f069c13e4d0 0x7f069c13e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f06a1bca640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069c0ac270 con 0x7f069c0703a0 2024-04-22T05:55:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f06a1bca640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c13ee80 con 0x7f069c13e4d0 2024-04-22T05:55:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f06a1bca640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c070f20 con 0x7f069c070780 2024-04-22T05:55:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f069b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f069c0703a0 0x7f069c06de40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36784/0 (socket says 172.21.15.40:36784) 2024-04-22T05:55:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f069bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f069c13e4d0 0x7f069c13e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f069b7fe640 1 -- 172.21.15.40:0/914463012 learned_addr learned my addr 172.21.15.40:0/914463012 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:55:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f069affd640 1 --2- 172.21.15.40:0/914463012 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f069c070780 0x7f069c141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f069a7fc640 1 -- 172.21.15.40:0/914463012 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195781784 0 0) 0x7f069c0ac270 con 0x7f069c0703a0 2024-04-22T05:55:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f069a7fc640 1 -- 172.21.15.40:0/914463012 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0684003530 con 0x7f069c0703a0 2024-04-22T05:55:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f069bfff640 1 -- 172.21.15.40:0/914463012 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f069c070780 msgr2=0x7f069c141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f069bfff640 1 --2- 172.21.15.40:0/914463012 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f069c070780 0x7f069c141c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f069bfff640 1 -- 172.21.15.40:0/914463012 >> v1:172.21.15.40:6789/0 conn(0x7f069c0703a0 legacy=0x7f069c06de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.710+0000 7f069bfff640 1 -- 172.21.15.40:0/914463012 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c141760 con 0x7f069c13e4d0 2024-04-22T05:55:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069bfff640 1 --2- 172.21.15.40:0/914463012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f069c13e4d0 0x7f069c13e940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f06940097c0 tx=0x7f069401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51241e486ff13449 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069a7fc640 1 -- 172.21.15.40:0/914463012 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06940ab610 con 0x7f069c13e4d0 2024-04-22T05:55:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069a7fc640 1 -- 172.21.15.40:0/914463012 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06940ab770 con 0x7f069c13e4d0 2024-04-22T05:55:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069a7fc640 1 -- 172.21.15.40:0/914463012 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06940abb00 con 0x7f069c13e4d0 2024-04-22T05:55:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 -- 172.21.15.40:0/914463012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f069c13e4d0 msgr2=0x7f069c13e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 --2- 172.21.15.40:0/914463012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f069c13e4d0 0x7f069c13e940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f06940097c0 tx=0x7f069401c3f0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 -- 172.21.15.40:0/914463012 shutdown_connections 2024-04-22T05:55:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 --2- 172.21.15.40:0/914463012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f069c13e4d0 0x7f069c13e940 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 --2- 172.21.15.40:0/914463012 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f069c070780 0x7f069c141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 -- 172.21.15.40:0/914463012 >> 172.21.15.40:0/914463012 conn(0x7f069c073f00 msgr2=0x7f069c0b7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 -- 172.21.15.40:0/914463012 shutdown_connections 2024-04-22T05:55:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 -- 172.21.15.40:0/914463012 wait complete. 2024-04-22T05:55:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 Processor -- start 2024-04-22T05:55:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 -- start start 2024-04-22T05:55:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f069c070780 0x7f069c139b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f069c13e4d0 0x7f069c13a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f069c1dc960 0x7f069c1dcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c138910 con 0x7f069c1dc960 2024-04-22T05:55:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c137f50 con 0x7f069c070780 2024-04-22T05:55:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c138110 con 0x7f069c13e4d0 2024-04-22T05:55:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f069c1dc960 0x7f069c1dcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f069c1dc960 0x7f069c1dcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34078/0 (socket says 172.21.15.40:34078) 2024-04-22T05:55:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069affd640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f069c13e4d0 0x7f069c13a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069bfff640 1 -- 172.21.15.40:0/561883334 learned_addr learned my addr 172.21.15.40:0/561883334 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069b7fe640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f069c070780 0x7f069c139b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069bfff640 1 -- 172.21.15.40:0/561883334 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f069c13e4d0 msgr2=0x7f069c13a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069bfff640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f069c13e4d0 0x7f069c13a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069bfff640 1 -- 172.21.15.40:0/561883334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f069c070780 msgr2=0x7f069c139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069bfff640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f069c070780 0x7f069c139b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069bfff640 1 -- 172.21.15.40:0/561883334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0694009420 con 0x7f069c1dc960 2024-04-22T05:55:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069b7fe640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f069c070780 0x7f069c139b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:55:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f069bfff640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f069c1dc960 0x7f069c1dcdf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f06940ad040 tx=0x7f069401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f0683fff640 1 -- 172.21.15.40:0/561883334 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06940bf610 con 0x7f069c1dc960 2024-04-22T05:55:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f0683fff640 1 -- 172.21.15.40:0/561883334 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06940bf770 con 0x7f069c1dc960 2024-04-22T05:55:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 -- 172.21.15.40:0/561883334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c12e0c0 con 0x7f069c1dc960 2024-04-22T05:55:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f0683fff640 1 -- 172.21.15.40:0/561883334 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06940bfb00 con 0x7f069c1dc960 2024-04-22T05:55:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.714+0000 7f06a1bca640 1 -- 172.21.15.40:0/561883334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c137bd0 con 0x7f069c1dc960 2024-04-22T05:55:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.718+0000 7f06a1bca640 1 -- 172.21.15.40:0/561883334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f069c1394e0 con 0x7f069c1dc960 2024-04-22T05:55:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.718+0000 7f0683fff640 1 -- 172.21.15.40:0/561883334 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f06940b5400 con 0x7f069c1dc960 2024-04-22T05:55:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.718+0000 7f0683fff640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f0688066690 0x7f0688068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.718+0000 7f0683fff640 1 -- 172.21.15.40:0/561883334 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f069411f1a0 con 0x7f069c1dc960 2024-04-22T05:55:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.718+0000 7f0683fff640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] conn(0x7f068806d7f0 0x7f068806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.718+0000 7f0683fff640 1 -- 172.21.15.40:0/561883334 --> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f06880702c0 con 0x7f068806d7f0 2024-04-22T05:55:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.718+0000 7f0683fff640 1 -- 172.21.15.40:0/561883334 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f069411f5f0 con 0x7f069c1dc960 2024-04-22T05:55:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.718+0000 7f069b7fe640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f0688066690 0x7f0688068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.718+0000 7f069affd640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] conn(0x7f068806d7f0 0x7f068806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.718+0000 7f069affd640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] conn(0x7f068806d7f0 0x7f068806fc10 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.718+0000 7f069b7fe640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f0688066690 0x7f0688068b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f068c001570 tx=0x7f068c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.718+0000 7f0683fff640 1 -- 172.21.15.40:0/561883334 <== osd.1 v2:172.21.15.40:6800/24984 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f06880702c0 con 0x7f068806d7f0 2024-04-22T05:55:21.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.730+0000 7f06a1bca640 1 -- 172.21.15.40:0/561883334 --> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f069c04e9c0 con 0x7f068806d7f0 2024-04-22T05:55:21.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.734+0000 7f0683fff640 1 -- 172.21.15.40:0/561883334 <== osd.1 v2:172.21.15.40:6800/24984 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f069c04e9c0 con 0x7f068806d7f0 2024-04-22T05:55:21.735 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-22T05:55:21.735 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T05:55:21.735 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-22T05:55:21.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.734+0000 7f06a1bca640 1 -- 172.21.15.40:0/561883334 >> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] conn(0x7f068806d7f0 msgr2=0x7f068806fc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.734+0000 7f06a1bca640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.40:6800/24984,v1:172.21.15.40:6801/24984] conn(0x7f068806d7f0 0x7f068806fc10 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.734+0000 7f06a1bca640 1 -- 172.21.15.40:0/561883334 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f0688066690 msgr2=0x7f0688068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.734+0000 7f06a1bca640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f0688066690 0x7f0688068b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f068c001570 tx=0x7f068c009040 comp rx=0 tx=0).stop 2024-04-22T05:55:21.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.734+0000 7f06a1bca640 1 -- 172.21.15.40:0/561883334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f069c1dc960 msgr2=0x7f069c1dcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.734+0000 7f06a1bca640 1 --2- 172.21.15.40:0/561883334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f069c1dc960 0x7f069c1dcdf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f06940ad040 tx=0x7f069401ac30 comp rx=0 tx=0).stop 2024-04-22T05:55:21.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.734+0000 7f069bfff640 1 -- 172.21.15.40:0/561883334 reap_dead start 2024-04-22T05:55:21.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.734+0000 7f06a1bca640 1 -- 172.21.15.40:0/561883334 shutdown_connections 2024-04-22T05:55:21.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.734+0000 7f06a1bca640 1 -- 172.21.15.40:0/561883334 >> 172.21.15.40:0/561883334 conn(0x7f069c0703a0 msgr2=0x7f069c0b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:21.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.734+0000 7f06a1bca640 1 -- 172.21.15.40:0/561883334 shutdown_connections 2024-04-22T05:55:21.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.734+0000 7f06a1bca640 1 -- 172.21.15.40:0/561883334 wait complete. 2024-04-22T05:55:21.751 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-22T05:55:21.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc502fe0640 1 Processor -- start 2024-04-22T05:55:21.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc502fe0640 1 -- start start 2024-04-22T05:55:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc502fe0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4fc06f510 0x7fc4fc06f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc502fe0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc4fc06fe90 0x7fc4fc14bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc502fe0640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4fc06b610 con 0x7fc4fc06b230 2024-04-22T05:55:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc502fe0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4fc071140 con 0x7fc4fc06f510 2024-04-22T05:55:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc502fe0640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4fc070a40 con 0x7fc4fc06fe90 2024-04-22T05:55:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc4fbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4fc06f510 0x7fc4fc06f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc4fbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4fc06f510 0x7fc4fc06f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34092/0 (socket says 172.21.15.40:34092) 2024-04-22T05:55:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc4fbfff640 1 -- 172.21.15.40:0/2388084826 learned_addr learned my addr 172.21.15.40:0/2388084826 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc50157d640 1 --2- 172.21.15.40:0/2388084826 >> v2:172.21.15.62:3300/0 conn(0x7fc4fc06fe90 0x7fc4fc14bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc4fbfff640 1 -- 172.21.15.40:0/2388084826 >> v2:172.21.15.62:3300/0 conn(0x7fc4fc06fe90 msgr2=0x7fc4fc14bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc4fbfff640 1 --2- 172.21.15.40:0/2388084826 >> v2:172.21.15.62:3300/0 conn(0x7fc4fc06fe90 0x7fc4fc14bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc4fbfff640 1 -- 172.21.15.40:0/2388084826 >> v1:172.21.15.62:6789/0 conn(0x7fc4fc06b230 legacy=0x7fc4fc140880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.866+0000 7fc4fbfff640 1 -- 172.21.15.40:0/2388084826 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc070bb0 con 0x7fc4fc06f510 2024-04-22T05:55:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc4fbfff640 1 --2- 172.21.15.40:0/2388084826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4fc06f510 0x7fc4fc06f950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc06bb70 tx=0x7fc4ec01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e068be80f7c68af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc4fb7fe640 1 -- 172.21.15.40:0/2388084826 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc071b80 con 0x7fc4fc06f510 2024-04-22T05:55:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc4fb7fe640 1 -- 172.21.15.40:0/2388084826 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec0a7050 con 0x7fc4fc06f510 2024-04-22T05:55:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc4fb7fe640 1 -- 172.21.15.40:0/2388084826 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec0ab820 con 0x7fc4fc06f510 2024-04-22T05:55:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 -- 172.21.15.40:0/2388084826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4fc06f510 msgr2=0x7fc4fc06f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 --2- 172.21.15.40:0/2388084826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4fc06f510 0x7fc4fc06f950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc06bb70 tx=0x7fc4ec01c430 comp rx=0 tx=0).stop 2024-04-22T05:55:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 -- 172.21.15.40:0/2388084826 shutdown_connections 2024-04-22T05:55:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 --2- 172.21.15.40:0/2388084826 >> v2:172.21.15.62:3300/0 conn(0x7fc4fc06fe90 0x7fc4fc14bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 --2- 172.21.15.40:0/2388084826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4fc06f510 0x7fc4fc06f950 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 -- 172.21.15.40:0/2388084826 >> 172.21.15.40:0/2388084826 conn(0x7fc4fc0b4320 msgr2=0x7fc4fc0b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 -- 172.21.15.40:0/2388084826 shutdown_connections 2024-04-22T05:55:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 -- 172.21.15.40:0/2388084826 wait complete. 2024-04-22T05:55:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 Processor -- start 2024-04-22T05:55:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 -- start start 2024-04-22T05:55:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4fc06b230 0x7fc4fc134060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4fc06f510 0x7fc4fc136bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc4fc06fe90 0x7fc4fc1d99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc1d9520 con 0x7fc4fc06f510 2024-04-22T05:55:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc0adb80 con 0x7fc4fc06b230 2024-04-22T05:55:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc502fe0640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc1d9140 con 0x7fc4fc06fe90 2024-04-22T05:55:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc50157d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc4fc06fe90 0x7fc4fc1d99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc50157d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc4fc06fe90 0x7fc4fc1d99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57536/0 (socket says 172.21.15.40:57536) 2024-04-22T05:55:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc50157d640 1 -- 172.21.15.40:0/192334243 learned_addr learned my addr 172.21.15.40:0/192334243 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc4fbfff640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4fc06f510 0x7fc4fc136bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc500d7c640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4fc06b230 0x7fc4fc134060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc50157d640 1 -- 172.21.15.40:0/192334243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4fc06b230 msgr2=0x7fc4fc134060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc50157d640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4fc06b230 0x7fc4fc134060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc50157d640 1 -- 172.21.15.40:0/192334243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4fc06f510 msgr2=0x7fc4fc136bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc50157d640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4fc06f510 0x7fc4fc136bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc50157d640 1 -- 172.21.15.40:0/192334243 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec0094d0 con 0x7fc4fc06fe90 2024-04-22T05:55:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.870+0000 7fc4fbfff640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4fc06f510 0x7fc4fc136bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.874+0000 7fc50157d640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc4fc06fe90 0x7fc4fc1d99b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4007e60 tx=0x7fc4f400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.874+0000 7fc4f97fa640 1 -- 172.21.15.40:0/192334243 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4014420 con 0x7fc4fc06fe90 2024-04-22T05:55:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.874+0000 7fc4f97fa640 1 -- 172.21.15.40:0/192334243 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f40145a0 con 0x7fc4fc06fe90 2024-04-22T05:55:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.874+0000 7fc502fe0640 1 -- 172.21.15.40:0/192334243 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4fc0ac9d0 con 0x7fc4fc06fe90 2024-04-22T05:55:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.874+0000 7fc4f97fa640 1 -- 172.21.15.40:0/192334243 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4014950 con 0x7fc4fc06fe90 2024-04-22T05:55:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.874+0000 7fc502fe0640 1 -- 172.21.15.40:0/192334243 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4fc137300 con 0x7fc4fc06fe90 2024-04-22T05:55:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.874+0000 7fc502fe0640 1 -- 172.21.15.40:0/192334243 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4fc1d9690 con 0x7fc4fc06fe90 2024-04-22T05:55:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.874+0000 7fc4f97fa640 1 -- 172.21.15.40:0/192334243 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc4f4014ab0 con 0x7fc4fc06fe90 2024-04-22T05:55:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.874+0000 7fc4f97fa640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fc4dc0666c0 0x7fc4dc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.874+0000 7fc4f97fa640 1 -- 172.21.15.40:0/192334243 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7fc4f4084910 con 0x7fc4fc06fe90 2024-04-22T05:55:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.874+0000 7fc500d7c640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fc4dc0666c0 0x7fc4dc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.874+0000 7fc500d7c640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fc4dc0666c0 0x7fc4dc068b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4007fd0 tx=0x7fc4e4007e40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:21.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:21.878+0000 7fc4f97fa640 1 -- 172.21.15.40:0/192334243 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7fc4f4051d60 con 0x7fc4fc06fe90 2024-04-22T05:55:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.034+0000 7fc502fe0640 1 -- 172.21.15.40:0/192334243 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4fc14ea10 con 0x7fc4dc0666c0 2024-04-22T05:55:22.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.034+0000 7fc4f97fa640 1 -- 172.21.15.40:0/192334243 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc4e4004c00 con 0x7fc4dc0666c0 2024-04-22T05:55:22.037 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 1 to deep-scrub 2024-04-22T05:55:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.038+0000 7fc502fe0640 1 -- 172.21.15.40:0/192334243 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fc4dc0666c0 msgr2=0x7fc4dc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.038+0000 7fc502fe0640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fc4dc0666c0 0x7fc4dc068b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4007fd0 tx=0x7fc4e4007e40 comp rx=0 tx=0).stop 2024-04-22T05:55:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.038+0000 7fc502fe0640 1 -- 172.21.15.40:0/192334243 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc4fc06fe90 msgr2=0x7fc4fc1d99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.038+0000 7fc502fe0640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc4fc06fe90 0x7fc4fc1d99b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4007e60 tx=0x7fc4f400e4b0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.038+0000 7fc502fe0640 1 -- 172.21.15.40:0/192334243 shutdown_connections 2024-04-22T05:55:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.038+0000 7fc502fe0640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fc4dc0666c0 0x7fc4dc068b80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.038+0000 7fc502fe0640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fc4fc06fe90 0x7fc4fc1d99b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.038+0000 7fc502fe0640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4fc06f510 0x7fc4fc136bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.038+0000 7fc502fe0640 1 --2- 172.21.15.40:0/192334243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4fc06b230 0x7fc4fc134060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.038+0000 7fc502fe0640 1 -- 172.21.15.40:0/192334243 >> 172.21.15.40:0/192334243 conn(0x7fc4fc0b4320 msgr2=0x7fc4fc13f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:22.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.038+0000 7fc502fe0640 1 -- 172.21.15.40:0/192334243 shutdown_connections 2024-04-22T05:55:22.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.038+0000 7fc502fe0640 1 -- 172.21.15.40:0/192334243 wait complete. 2024-04-22T05:55:22.056 INFO:tasks.ceph:Scrubbing osd.2 2024-04-22T05:55:22.056 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-22T05:55:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f78047640 1 Processor -- start 2024-04-22T05:55:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f78047640 1 -- start start 2024-04-22T05:55:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f78047640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9f700a7e90 0x7f9f7006df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f78047640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9f7006e480 0x7f9f7013c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f78047640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f7013f900 con 0x7f9f700a7ab0 2024-04-22T05:55:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f78047640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f7006e8c0 con 0x7f9f700a7e90 2024-04-22T05:55:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f78047640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f700a8d00 con 0x7f9f7006e480 2024-04-22T05:55:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f755e2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9f700a7e90 0x7f9f7006df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f755e2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9f700a7e90 0x7f9f7006df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34112/0 (socket says 172.21.15.40:34112) 2024-04-22T05:55:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f75de3640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9f700a7ab0 0x7f9f7013fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36796/0 (socket says 172.21.15.40:36796) 2024-04-22T05:55:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f755e2640 1 -- 172.21.15.40:0/4127501958 learned_addr learned my addr 172.21.15.40:0/4127501958 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f765e4640 1 --2- 172.21.15.40:0/4127501958 >> v2:172.21.15.62:3300/0 conn(0x7f9f7006e480 0x7f9f7013c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f755e2640 1 -- 172.21.15.40:0/4127501958 >> v2:172.21.15.62:3300/0 conn(0x7f9f7006e480 msgr2=0x7f9f7013c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f755e2640 1 --2- 172.21.15.40:0/4127501958 >> v2:172.21.15.62:3300/0 conn(0x7f9f7006e480 0x7f9f7013c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f755e2640 1 -- 172.21.15.40:0/4127501958 >> v1:172.21.15.40:6789/0 conn(0x7f9f700a7ab0 legacy=0x7f9f7013fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f755e2640 1 -- 172.21.15.40:0/4127501958 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f700a8f80 con 0x7f9f700a7e90 2024-04-22T05:55:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.174+0000 7f9f765e4640 1 --2- 172.21.15.40:0/4127501958 >> v2:172.21.15.62:3300/0 conn(0x7f9f7006e480 0x7f9f7013c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:55:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f755e2640 1 --2- 172.21.15.40:0/4127501958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f700a7e90 0x7f9f7006df40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9f700afa40 tx=0x7f9f6001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd7f09ec08d5ba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f74de1640 1 -- 172.21.15.40:0/4127501958 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f600aa5f0 con 0x7f9f700a7e90 2024-04-22T05:55:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f74de1640 1 -- 172.21.15.40:0/4127501958 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f600aa750 con 0x7f9f700a7e90 2024-04-22T05:55:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f74de1640 1 -- 172.21.15.40:0/4127501958 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f600aaae0 con 0x7f9f700a7e90 2024-04-22T05:55:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 -- 172.21.15.40:0/4127501958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f700a7e90 msgr2=0x7f9f7006df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 --2- 172.21.15.40:0/4127501958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f700a7e90 0x7f9f7006df40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9f700afa40 tx=0x7f9f6001c470 comp rx=0 tx=0).stop 2024-04-22T05:55:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 -- 172.21.15.40:0/4127501958 shutdown_connections 2024-04-22T05:55:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 --2- 172.21.15.40:0/4127501958 >> v2:172.21.15.62:3300/0 conn(0x7f9f7006e480 0x7f9f7013c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 --2- 172.21.15.40:0/4127501958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f700a7e90 0x7f9f7006df40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 -- 172.21.15.40:0/4127501958 >> 172.21.15.40:0/4127501958 conn(0x7f9f700736e0 msgr2=0x7f9f70073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 -- 172.21.15.40:0/4127501958 shutdown_connections 2024-04-22T05:55:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 -- 172.21.15.40:0/4127501958 wait complete. 2024-04-22T05:55:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 Processor -- start 2024-04-22T05:55:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 -- start start 2024-04-22T05:55:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f7006e480 0x7f9f70143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f700a7ab0 0x7f9f7013ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f700a7e90 0x7f9f701386d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f701337b0 con 0x7f9f7006e480 2024-04-22T05:55:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7013b2a0 con 0x7f9f700a7e90 2024-04-22T05:55:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f78047640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f70144c40 con 0x7f9f700a7ab0 2024-04-22T05:55:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.178+0000 7f9f75de3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f7006e480 0x7f9f70143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f755e2640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f700a7ab0 0x7f9f7013ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f75de3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f7006e480 0x7f9f70143b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34124/0 (socket says 172.21.15.40:34124) 2024-04-22T05:55:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f75de3640 1 -- 172.21.15.40:0/3844807320 learned_addr learned my addr 172.21.15.40:0/3844807320 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f765e4640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f700a7e90 0x7f9f701386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f75de3640 1 -- 172.21.15.40:0/3844807320 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f700a7ab0 msgr2=0x7f9f7013ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f75de3640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f700a7ab0 0x7f9f7013ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f75de3640 1 -- 172.21.15.40:0/3844807320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f700a7e90 msgr2=0x7f9f701386d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f75de3640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f700a7e90 0x7f9f701386d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f75de3640 1 -- 172.21.15.40:0/3844807320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f600094d0 con 0x7f9f7006e480 2024-04-22T05:55:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f765e4640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f700a7e90 0x7f9f701386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:55:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f755e2640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f700a7ab0 0x7f9f7013ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:22.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f75de3640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f7006e480 0x7f9f70143b40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c490 tx=0x7f9f6c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f667fc640 1 -- 172.21.15.40:0/3844807320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c007480 con 0x7f9f7006e480 2024-04-22T05:55:22.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f667fc640 1 -- 172.21.15.40:0/3844807320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c0075e0 con 0x7f9f7006e480 2024-04-22T05:55:22.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f78047640 1 -- 172.21.15.40:0/3844807320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f70144f20 con 0x7f9f7006e480 2024-04-22T05:55:22.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f667fc640 1 -- 172.21.15.40:0/3844807320 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c004a10 con 0x7f9f7006e480 2024-04-22T05:55:22.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f78047640 1 -- 172.21.15.40:0/3844807320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f70138c10 con 0x7f9f7006e480 2024-04-22T05:55:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f78047640 1 -- 172.21.15.40:0/3844807320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9f70138e60 con 0x7f9f7006e480 2024-04-22T05:55:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f667fc640 1 -- 172.21.15.40:0/3844807320 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9f6c004b70 con 0x7f9f7006e480 2024-04-22T05:55:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f667fc640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9f54066690 0x7f9f54068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f667fc640 1 -- 172.21.15.40:0/3844807320 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f9f6c083450 con 0x7f9f7006e480 2024-04-22T05:55:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f667fc640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] conn(0x7f9f5406d7f0 0x7f9f5406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f667fc640 1 -- 172.21.15.40:0/3844807320 --> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f540702c0 con 0x7f9f5406d7f0 2024-04-22T05:55:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f667fc640 1 -- 172.21.15.40:0/3844807320 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f6c085b10 con 0x7f9f7006e480 2024-04-22T05:55:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f755e2640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9f54066690 0x7f9f54068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f765e4640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] conn(0x7f9f5406d7f0 0x7f9f5406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f765e4640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] conn(0x7f9f5406d7f0 0x7f9f5406fc10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f755e2640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9f54066690 0x7f9f54068b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9f600ae000 tx=0x7f9f600a7000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.182+0000 7f9f667fc640 1 -- 172.21.15.40:0/3844807320 <== osd.2 v2:172.21.15.40:6808/24986 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f9f540702c0 con 0x7f9f5406d7f0 2024-04-22T05:55:22.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.198+0000 7f9f78047640 1 -- 172.21.15.40:0/3844807320 --> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9f70133610 con 0x7f9f5406d7f0 2024-04-22T05:55:22.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.198+0000 7f9f667fc640 1 -- 172.21.15.40:0/3844807320 <== osd.2 v2:172.21.15.40:6808/24986 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9f6800d510 con 0x7f9f5406d7f0 2024-04-22T05:55:22.200 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-22T05:55:22.200 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T05:55:22.200 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-22T05:55:22.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.198+0000 7f9f78047640 1 -- 172.21.15.40:0/3844807320 >> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] conn(0x7f9f5406d7f0 msgr2=0x7f9f5406fc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.198+0000 7f9f78047640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.40:6808/24986,v1:172.21.15.40:6809/24986] conn(0x7f9f5406d7f0 0x7f9f5406fc10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.198+0000 7f9f78047640 1 -- 172.21.15.40:0/3844807320 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9f54066690 msgr2=0x7f9f54068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.198+0000 7f9f78047640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9f54066690 0x7f9f54068b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9f600ae000 tx=0x7f9f600a7000 comp rx=0 tx=0).stop 2024-04-22T05:55:22.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.198+0000 7f9f78047640 1 -- 172.21.15.40:0/3844807320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f7006e480 msgr2=0x7f9f70143b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.198+0000 7f9f78047640 1 --2- 172.21.15.40:0/3844807320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f7006e480 0x7f9f70143b40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c490 tx=0x7f9f6c00c950 comp rx=0 tx=0).stop 2024-04-22T05:55:22.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.198+0000 7f9f765e4640 1 -- 172.21.15.40:0/3844807320 reap_dead start 2024-04-22T05:55:22.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.198+0000 7f9f78047640 1 -- 172.21.15.40:0/3844807320 shutdown_connections 2024-04-22T05:55:22.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.198+0000 7f9f78047640 1 -- 172.21.15.40:0/3844807320 >> 172.21.15.40:0/3844807320 conn(0x7f9f700736e0 msgr2=0x7f9f70133e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:22.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.198+0000 7f9f78047640 1 -- 172.21.15.40:0/3844807320 shutdown_connections 2024-04-22T05:55:22.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.198+0000 7f9f78047640 1 -- 172.21.15.40:0/3844807320 wait complete. 2024-04-22T05:55:22.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-22T05:55:22.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9e033c640 1 Processor -- start 2024-04-22T05:55:22.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9e033c640 1 -- start start 2024-04-22T05:55:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9e033c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff9d806df40 0x7ff9d8145ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9e033c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d8145a10 con 0x7ff9d81463e0 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9e033c640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d8071890 con 0x7ff9d80714b0 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9e033c640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d806e3b0 con 0x7ff9d806df40 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9de8d9640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff9d81463e0 0x7ff9d813f1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36804/0 (socket says 172.21.15.40:36804) 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9de8d9640 1 -- 172.21.15.40:0/3000303052 learned_addr learned my addr 172.21.15.40:0/3000303052 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd8d7640 1 --2- 172.21.15.40:0/3000303052 >> v2:172.21.15.62:3300/0 conn(0x7ff9d806df40 0x7ff9d8145ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581451175 0 0) 0x7ff9d8145a10 con 0x7ff9d81463e0 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9c0003530 con 0x7ff9d81463e0 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2082550762 0 0) 0x7ff9c0003530 con 0x7ff9d81463e0 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 >> v2:172.21.15.62:3300/0 conn(0x7ff9d806df40 msgr2=0x7ff9d8145ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 --2- 172.21.15.40:0/3000303052 >> v2:172.21.15.62:3300/0 conn(0x7ff9d806df40 0x7ff9d8145ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 >> v1:172.21.15.62:6789/0 conn(0x7ff9d80714b0 legacy=0x7ff9d8070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d80a7bb0 con 0x7ff9d81463e0 2024-04-22T05:55:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2268782488 0 0) 0x7ff9c8001680 con 0x7ff9d81463e0 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd8d7640 1 --2- 172.21.15.40:0/3000303052 >> v2:172.21.15.62:3300/0 conn(0x7ff9d806df40 0x7ff9d8145ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 >> v1:172.21.15.40:6789/0 conn(0x7ff9d81463e0 legacy=0x7ff9d813f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 --2- 172.21.15.40:0/3000303052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c0003980 0x7ff9c0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 --2- 172.21.15.40:0/3000303052 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9c00047d0 0x7ff9c0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 --2- 172.21.15.40:0/3000303052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9c0005190 0x7ff9c000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d806e3b0 con 0x7ff9c0003980 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c000c130 con 0x7ff9c0005190 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c000c2f0 con 0x7ff9c00047d0 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd8d7640 1 --2- 172.21.15.40:0/3000303052 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9c00047d0 0x7ff9c0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9de0d8640 1 --2- 172.21.15.40:0/3000303052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c0003980 0x7ff9c0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9de8d9640 1 --2- 172.21.15.40:0/3000303052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9c0005190 0x7ff9c000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd8d7640 1 -- 172.21.15.40:0/3000303052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9c0005190 msgr2=0x7ff9c000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd8d7640 1 --2- 172.21.15.40:0/3000303052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9c0005190 0x7ff9c000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd8d7640 1 -- 172.21.15.40:0/3000303052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c0003980 msgr2=0x7ff9c0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:22.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd8d7640 1 --2- 172.21.15.40:0/3000303052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c0003980 0x7ff9c0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9dd8d7640 1 -- 172.21.15.40:0/3000303052 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c0003530 con 0x7ff9c00047d0 2024-04-22T05:55:22.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9de8d9640 1 -- 172.21.15.40:0/3000303052 reap_dead start 2024-04-22T05:55:22.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.326+0000 7ff9de0d8640 1 --2- 172.21.15.40:0/3000303052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c0003980 0x7ff9c0003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9dd8d7640 1 --2- 172.21.15.40:0/3000303052 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9c00047d0 0x7ff9c0004c50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00e350 tx=0x7ff9cc01cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f842adcbe23971c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0a5cb0 con 0x7ff9c00047d0 2024-04-22T05:55:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc00f040 con 0x7ff9c00047d0 2024-04-22T05:55:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9dd0d6640 1 -- 172.21.15.40:0/3000303052 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0ad640 con 0x7ff9c00047d0 2024-04-22T05:55:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 -- 172.21.15.40:0/3000303052 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9c00047d0 msgr2=0x7ff9c0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 --2- 172.21.15.40:0/3000303052 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9c00047d0 0x7ff9c0004c50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00e350 tx=0x7ff9cc01cdb0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 -- 172.21.15.40:0/3000303052 shutdown_connections 2024-04-22T05:55:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 --2- 172.21.15.40:0/3000303052 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9c00047d0 0x7ff9c0004c50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 -- 172.21.15.40:0/3000303052 >> 172.21.15.40:0/3000303052 conn(0x7ff9d80b3170 msgr2=0x7ff9d80b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 -- 172.21.15.40:0/3000303052 shutdown_connections 2024-04-22T05:55:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 -- 172.21.15.40:0/3000303052 wait complete. 2024-04-22T05:55:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 Processor -- start 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 -- start start 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9c00047d0 0x7ff9d81404d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d81d39b0 0x7ff9d8141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9d81e1370 0x7ff9d81e1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d813b080 con 0x7ff9d81d39b0 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d813ae30 con 0x7ff9c00047d0 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.330+0000 7ff9e033c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d8142760 con 0x7ff9d81e1370 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9dd8d7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d81d39b0 0x7ff9d8141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9de8d9640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9d81e1370 0x7ff9d81e1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9dd8d7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d81d39b0 0x7ff9d8141080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34134/0 (socket says 172.21.15.40:34134) 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9dd8d7640 1 -- 172.21.15.40:0/1574424017 learned_addr learned my addr 172.21.15.40:0/1574424017 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9de0d8640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9c00047d0 0x7ff9d81404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9dd8d7640 1 -- 172.21.15.40:0/1574424017 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9d81e1370 msgr2=0x7ff9d81e1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9dd8d7640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9d81e1370 0x7ff9d81e1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9dd8d7640 1 -- 172.21.15.40:0/1574424017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9c00047d0 msgr2=0x7ff9d81404d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9dd8d7640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9c00047d0 0x7ff9d81404d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9dd8d7640 1 -- 172.21.15.40:0/1574424017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d8142a40 con 0x7ff9d81d39b0 2024-04-22T05:55:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9de8d9640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9d81e1370 0x7ff9d81e1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9dd8d7640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d81d39b0 0x7ff9d8141080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc01cbe0 tx=0x7ff9cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/1574424017 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0ba070 con 0x7ff9d81d39b0 2024-04-22T05:55:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/1574424017 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc0ad8d0 con 0x7ff9d81d39b0 2024-04-22T05:55:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/1574424017 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0adca0 con 0x7ff9d81d39b0 2024-04-22T05:55:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9e033c640 1 -- 172.21.15.40:0/1574424017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d81d5790 con 0x7ff9d81d39b0 2024-04-22T05:55:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9e033c640 1 -- 172.21.15.40:0/1574424017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d81415c0 con 0x7ff9d81d39b0 2024-04-22T05:55:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9e033c640 1 -- 172.21.15.40:0/1574424017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d81417b0 con 0x7ff9d81d39b0 2024-04-22T05:55:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/1574424017 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9cc0b4040 con 0x7ff9d81d39b0 2024-04-22T05:55:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9d6ffd640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff9c4066690 0x7ff9c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/1574424017 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7ff9cc11eb00 con 0x7ff9d81d39b0 2024-04-22T05:55:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.334+0000 7ff9de0d8640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff9c4066690 0x7ff9c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.338+0000 7ff9de0d8640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff9c4066690 0x7ff9c4068b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0000c10 tx=0x7ff9d0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.338+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/1574424017 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7ff9cc0ec020 con 0x7ff9d81d39b0 2024-04-22T05:55:22.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9e033c640 1 -- 172.21.15.40:0/1574424017 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9d813fbc0 con 0x7ff9c4066690 2024-04-22T05:55:22.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9d6ffd640 1 -- 172.21.15.40:0/1574424017 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff9d00036c0 con 0x7ff9c4066690 2024-04-22T05:55:22.500 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 2 to deep-scrub 2024-04-22T05:55:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9e033c640 1 -- 172.21.15.40:0/1574424017 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff9c4066690 msgr2=0x7ff9c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9e033c640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff9c4066690 0x7ff9c4068b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0000c10 tx=0x7ff9d0009040 comp rx=0 tx=0).stop 2024-04-22T05:55:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9e033c640 1 -- 172.21.15.40:0/1574424017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d81d39b0 msgr2=0x7ff9d8141080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9e033c640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d81d39b0 0x7ff9d8141080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc01cbe0 tx=0x7ff9cc009750 comp rx=0 tx=0).stop 2024-04-22T05:55:22.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9e033c640 1 -- 172.21.15.40:0/1574424017 shutdown_connections 2024-04-22T05:55:22.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9e033c640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7ff9c4066690 0x7ff9c4068b50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9e033c640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7ff9d81e1370 0x7ff9d81e1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9e033c640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9d81d39b0 0x7ff9d8141080 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9e033c640 1 --2- 172.21.15.40:0/1574424017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9c00047d0 0x7ff9d81404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9e033c640 1 -- 172.21.15.40:0/1574424017 >> 172.21.15.40:0/1574424017 conn(0x7ff9d80b3170 msgr2=0x7ff9d80b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:22.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9e033c640 1 -- 172.21.15.40:0/1574424017 shutdown_connections 2024-04-22T05:55:22.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.498+0000 7ff9e033c640 1 -- 172.21.15.40:0/1574424017 wait complete. 2024-04-22T05:55:22.518 INFO:tasks.ceph:Scrubbing osd.3 2024-04-22T05:55:22.518 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-22T05:55:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.642+0000 7fef56597640 1 Processor -- start 2024-04-22T05:55:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.642+0000 7fef56597640 1 -- start start 2024-04-22T05:55:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef56597640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef50071b50 0x7fef501451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef56597640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fef50140860 0x7fef50140c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef56597640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef50070c90 con 0x7fef50144b90 2024-04-22T05:55:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef56597640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef501411d0 con 0x7fef50071b50 2024-04-22T05:55:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef56597640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef50144f70 con 0x7fef50140860 2024-04-22T05:55:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef54b34640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fef50140860 0x7fef50140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef4f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fef50071b50 0x7fef501451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef4ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fef50144b90 0x7fef5006d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36812/0 (socket says 172.21.15.40:36812) 2024-04-22T05:55:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef4ffff640 1 -- 172.21.15.40:0/37379140 learned_addr learned my addr 172.21.15.40:0/37379140 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:55:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef54b34640 1 -- 172.21.15.40:0/37379140 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fef50140860 msgr2=0x7fef50140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T05:55:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef54b34640 1 -- 172.21.15.40:0/37379140 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fef50140860 msgr2=0x7fef50140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T05:55:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef54b34640 1 --2- 172.21.15.40:0/37379140 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fef50140860 0x7fef50140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T05:55:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef54b34640 1 --2- 172.21.15.40:0/37379140 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fef50140860 0x7fef50140c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T05:55:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef4f7fe640 1 -- 172.21.15.40:0/37379140 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fef50140860 msgr2=0x7fef50140c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T05:55:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef4f7fe640 1 --2- 172.21.15.40:0/37379140 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fef50140860 0x7fef50140c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef4f7fe640 1 -- 172.21.15.40:0/37379140 >> v1:172.21.15.40:6789/0 conn(0x7fef50144b90 legacy=0x7fef5006d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef4f7fe640 1 -- 172.21.15.40:0/37379140 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef50071300 con 0x7fef50071b50 2024-04-22T05:55:22.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef4effd640 1 -- 172.21.15.40:0/37379140 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506271463 0 0) 0x7fef50070c90 con 0x7fef50144b90 2024-04-22T05:55:22.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.646+0000 7fef4f7fe640 1 --2- 172.21.15.40:0/37379140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef50071b50 0x7fef501451e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fef440097c0 tx=0x7fef4401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3951087a88c3e100 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4effd640 1 -- 172.21.15.40:0/37379140 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef440ab600 con 0x7fef50071b50 2024-04-22T05:55:22.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4effd640 1 -- 172.21.15.40:0/37379140 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef440a7050 con 0x7fef50071b50 2024-04-22T05:55:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4effd640 1 -- 172.21.15.40:0/37379140 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef440ab980 con 0x7fef50071b50 2024-04-22T05:55:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 -- 172.21.15.40:0/37379140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef50071b50 msgr2=0x7fef501451e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 --2- 172.21.15.40:0/37379140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef50071b50 0x7fef501451e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fef440097c0 tx=0x7fef4401c3f0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 -- 172.21.15.40:0/37379140 shutdown_connections 2024-04-22T05:55:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 --2- 172.21.15.40:0/37379140 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fef50140860 0x7fef50140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 --2- 172.21.15.40:0/37379140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef50071b50 0x7fef501451e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 -- 172.21.15.40:0/37379140 >> 172.21.15.40:0/37379140 conn(0x7fef500b5560 msgr2=0x7fef500b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:22.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 -- 172.21.15.40:0/37379140 shutdown_connections 2024-04-22T05:55:22.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 -- 172.21.15.40:0/37379140 wait complete. 2024-04-22T05:55:22.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 Processor -- start 2024-04-22T05:55:22.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 -- start start 2024-04-22T05:55:22.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef50071b50 0x7fef50134ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fef50140860 0x7fef5012e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef50144b90 0x7fef5012e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5012ee20 con 0x7fef50071b50 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef50136710 con 0x7fef50144b90 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef50135200 con 0x7fef50140860 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef50071b50 0x7fef50134ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef50071b50 0x7fef50134ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34148/0 (socket says 172.21.15.40:34148) 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4ffff640 1 -- 172.21.15.40:0/740982037 learned_addr learned my addr 172.21.15.40:0/740982037 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4f7fe640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fef50140860 0x7fef5012e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef54b34640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef50144b90 0x7fef5012e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4ffff640 1 -- 172.21.15.40:0/740982037 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fef50140860 msgr2=0x7fef5012e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4ffff640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fef50140860 0x7fef5012e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4ffff640 1 -- 172.21.15.40:0/740982037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef50144b90 msgr2=0x7fef5012e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4ffff640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef50144b90 0x7fef5012e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4ffff640 1 -- 172.21.15.40:0/740982037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef44009420 con 0x7fef50071b50 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef54b34640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef50144b90 0x7fef5012e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:55:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4ffff640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef50071b50 0x7fef50134ba0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef400011f0 tx=0x7fef4000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4cff9640 1 -- 172.21.15.40:0/740982037 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef400074e0 con 0x7fef50071b50 2024-04-22T05:55:22.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4cff9640 1 -- 172.21.15.40:0/740982037 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef40007640 con 0x7fef50071b50 2024-04-22T05:55:22.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 -- 172.21.15.40:0/740982037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5012f010 con 0x7fef50071b50 2024-04-22T05:55:22.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef4cff9640 1 -- 172.21.15.40:0/740982037 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40004a10 con 0x7fef50071b50 2024-04-22T05:55:22.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.650+0000 7fef56597640 1 -- 172.21.15.40:0/740982037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef50144350 con 0x7fef50071b50 2024-04-22T05:55:22.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.654+0000 7fef56597640 1 -- 172.21.15.40:0/740982037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef5004e9c0 con 0x7fef50071b50 2024-04-22T05:55:22.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.654+0000 7fef4cff9640 1 -- 172.21.15.40:0/740982037 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fef4001b360 con 0x7fef50071b50 2024-04-22T05:55:22.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.654+0000 7fef4cff9640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fef3c066690 0x7fef3c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.654+0000 7fef4cff9640 1 -- 172.21.15.40:0/740982037 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7fef400839f0 con 0x7fef50071b50 2024-04-22T05:55:22.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.654+0000 7fef4cff9640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] conn(0x7fef3c06d7f0 0x7fef3c06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.654+0000 7fef4cff9640 1 -- 172.21.15.40:0/740982037 --> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef3c0702c0 con 0x7fef3c06d7f0 2024-04-22T05:55:22.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.654+0000 7fef4cff9640 1 -- 172.21.15.40:0/740982037 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef40083e40 con 0x7fef50071b50 2024-04-22T05:55:22.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.654+0000 7fef54b34640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] conn(0x7fef3c06d7f0 0x7fef3c06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.654+0000 7fef4f7fe640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fef3c066690 0x7fef3c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.654+0000 7fef54b34640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] conn(0x7fef3c06d7f0 0x7fef3c06fc10 crc :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.654+0000 7fef4f7fe640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fef3c066690 0x7fef3c068b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fef440ad000 tx=0x7fef44008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.654+0000 7fef4cff9640 1 -- 172.21.15.40:0/740982037 <== osd.3 v2:172.21.15.40:6817/24985 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fef3c0702c0 con 0x7fef3c06d7f0 2024-04-22T05:55:22.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.666+0000 7fef56597640 1 -- 172.21.15.40:0/740982037 --> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fef50135420 con 0x7fef3c06d7f0 2024-04-22T05:55:22.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.666+0000 7fef4cff9640 1 -- 172.21.15.40:0/740982037 <== osd.3 v2:172.21.15.40:6817/24985 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fef50135420 con 0x7fef3c06d7f0 2024-04-22T05:55:22.670 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-22T05:55:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T05:55:22.670 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-22T05:55:22.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.666+0000 7fef56597640 1 -- 172.21.15.40:0/740982037 >> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] conn(0x7fef3c06d7f0 msgr2=0x7fef3c06fc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.666+0000 7fef56597640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.40:6817/24985,v1:172.21.15.40:6819/24985] conn(0x7fef3c06d7f0 0x7fef3c06fc10 crc :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.666+0000 7fef56597640 1 -- 172.21.15.40:0/740982037 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fef3c066690 msgr2=0x7fef3c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.666+0000 7fef56597640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fef3c066690 0x7fef3c068b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fef440ad000 tx=0x7fef44008040 comp rx=0 tx=0).stop 2024-04-22T05:55:22.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.666+0000 7fef56597640 1 -- 172.21.15.40:0/740982037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef50071b50 msgr2=0x7fef50134ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.666+0000 7fef56597640 1 --2- 172.21.15.40:0/740982037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef50071b50 0x7fef50134ba0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef400011f0 tx=0x7fef4000c9b0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.666+0000 7fef54b34640 1 -- 172.21.15.40:0/740982037 reap_dead start 2024-04-22T05:55:22.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.666+0000 7fef56597640 1 -- 172.21.15.40:0/740982037 shutdown_connections 2024-04-22T05:55:22.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.666+0000 7fef56597640 1 -- 172.21.15.40:0/740982037 >> 172.21.15.40:0/740982037 conn(0x7fef500b5560 msgr2=0x7fef500b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:22.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.666+0000 7fef56597640 1 -- 172.21.15.40:0/740982037 shutdown_connections 2024-04-22T05:55:22.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.666+0000 7fef56597640 1 -- 172.21.15.40:0/740982037 wait complete. 2024-04-22T05:55:22.686 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-22T05:55:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4177fcb640 1 Processor -- start 2024-04-22T05:55:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4177fcb640 1 -- start start 2024-04-22T05:55:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4177fcb640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f41700a79a0 0x7f41700a7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4177fcb640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f417013e730 0x7f417013eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4177fcb640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417013f8b0 con 0x7f417006df40 2024-04-22T05:55:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4177fcb640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41700a8bb0 con 0x7f417013e730 2024-04-22T05:55:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4177fcb640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41700a8cf0 con 0x7f41700a79a0 2024-04-22T05:55:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4175d67640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f41700a79a0 0x7f41700a7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4175566640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f417006df40 0x7f417013fdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36818/0 (socket says 172.21.15.40:36818) 2024-04-22T05:55:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4175566640 1 -- 172.21.15.40:0/3016411762 learned_addr learned my addr 172.21.15.40:0/3016411762 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:55:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4176568640 1 --2- 172.21.15.40:0/3016411762 >> v2:172.21.15.62:3300/0 conn(0x7f417013e730 0x7f417013eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4175d67640 1 -- 172.21.15.40:0/3016411762 >> v2:172.21.15.62:3300/0 conn(0x7f417013e730 msgr2=0x7f417013eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:22.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4175d67640 1 --2- 172.21.15.40:0/3016411762 >> v2:172.21.15.62:3300/0 conn(0x7f417013e730 0x7f417013eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4174d65640 1 -- 172.21.15.40:0/3016411762 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406795738 0 0) 0x7f417013f8b0 con 0x7f417006df40 2024-04-22T05:55:22.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4175d67640 1 -- 172.21.15.40:0/3016411762 >> v1:172.21.15.40:6789/0 conn(0x7f417006df40 legacy=0x7f417013fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.798+0000 7f4175d67640 1 -- 172.21.15.40:0/3016411762 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41700a8e70 con 0x7f41700a79a0 2024-04-22T05:55:22.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4175d67640 1 --2- 172.21.15.40:0/3016411762 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f41700a79a0 0x7f41700a7d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f416c0097c0 tx=0x7f416c01c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3568e8701f9cef server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4174d65640 1 -- 172.21.15.40:0/3016411762 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c0ab600 con 0x7f41700a79a0 2024-04-22T05:55:22.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4174d65640 1 -- 172.21.15.40:0/3016411762 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c0a7050 con 0x7f41700a79a0 2024-04-22T05:55:22.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4174d65640 1 -- 172.21.15.40:0/3016411762 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c0ab980 con 0x7f41700a79a0 2024-04-22T05:55:22.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4177fcb640 1 -- 172.21.15.40:0/3016411762 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f41700a79a0 msgr2=0x7f41700a7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4177fcb640 1 --2- 172.21.15.40:0/3016411762 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f41700a79a0 0x7f41700a7d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f416c0097c0 tx=0x7f416c01c360 comp rx=0 tx=0).stop 2024-04-22T05:55:22.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4177fcb640 1 -- 172.21.15.40:0/3016411762 shutdown_connections 2024-04-22T05:55:22.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4177fcb640 1 --2- 172.21.15.40:0/3016411762 >> v2:172.21.15.62:3300/0 conn(0x7f417013e730 0x7f417013eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4177fcb640 1 --2- 172.21.15.40:0/3016411762 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f41700a79a0 0x7f41700a7d80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4177fcb640 1 -- 172.21.15.40:0/3016411762 >> 172.21.15.40:0/3016411762 conn(0x7f41700b3170 msgr2=0x7f41700b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:22.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4177fcb640 1 -- 172.21.15.40:0/3016411762 shutdown_connections 2024-04-22T05:55:22.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4177fcb640 1 -- 172.21.15.40:0/3016411762 wait complete. 2024-04-22T05:55:22.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4177fcb640 1 Processor -- start 2024-04-22T05:55:22.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.802+0000 7f4177fcb640 1 -- start start 2024-04-22T05:55:22.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4177fcb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f417006df40 0x7f417012e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4177fcb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700a79a0 0x7f41701d91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4177fcb640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f417013e730 0x7f417006ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4177fcb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41701d9b40 con 0x7f41700a79a0 2024-04-22T05:55:22.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4177fcb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417012ec50 con 0x7f417006df40 2024-04-22T05:55:22.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4177fcb640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417012ee10 con 0x7f417013e730 2024-04-22T05:55:22.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4176568640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f417013e730 0x7f417006ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4176568640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f417013e730 0x7f417006ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57600/0 (socket says 172.21.15.40:57600) 2024-04-22T05:55:22.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4176568640 1 -- 172.21.15.40:0/744758329 learned_addr learned my addr 172.21.15.40:0/744758329 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4175566640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700a79a0 0x7f41701d91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4175d67640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f417006df40 0x7f417012e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4176568640 1 -- 172.21.15.40:0/744758329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f417006df40 msgr2=0x7f417012e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4176568640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f417006df40 0x7f417012e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4176568640 1 -- 172.21.15.40:0/744758329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700a79a0 msgr2=0x7f41701d91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4176568640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700a79a0 0x7f41701d91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4176568640 1 -- 172.21.15.40:0/744758329 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c009420 con 0x7f417013e730 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4175d67640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f417006df40 0x7f417012e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4176568640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f417013e730 0x7f417006ff60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4168007e30 tx=0x7f416800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f41667fc640 1 -- 172.21.15.40:0/744758329 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168014620 con 0x7f417013e730 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f41667fc640 1 -- 172.21.15.40:0/744758329 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416800ee90 con 0x7f417013e730 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4177fcb640 1 -- 172.21.15.40:0/744758329 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41701d9720 con 0x7f417013e730 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f41667fc640 1 -- 172.21.15.40:0/744758329 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41680149c0 con 0x7f417013e730 2024-04-22T05:55:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4177fcb640 1 -- 172.21.15.40:0/744758329 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41701da230 con 0x7f417013e730 2024-04-22T05:55:22.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4177fcb640 1 -- 172.21.15.40:0/744758329 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4170067380 con 0x7f417013e730 2024-04-22T05:55:22.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f41667fc640 1 -- 172.21.15.40:0/744758329 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4168014b20 con 0x7f417013e730 2024-04-22T05:55:22.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f41667fc640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4154066690 0x7f4154068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:22.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f41667fc640 1 -- 172.21.15.40:0/744758329 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f4168084a20 con 0x7f417013e730 2024-04-22T05:55:22.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.806+0000 7f4175d67640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4154066690 0x7f4154068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:22.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.810+0000 7f4175d67640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4154066690 0x7f4154068b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f416c0ad000 tx=0x7f416c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:22.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.810+0000 7f41667fc640 1 -- 172.21.15.40:0/744758329 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f4168051e60 con 0x7f417013e730 2024-04-22T05:55:22.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.974+0000 7f4177fcb640 1 -- 172.21.15.40:0/744758329 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f417006e550 con 0x7f4154066690 2024-04-22T05:55:22.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.974+0000 7f41667fc640 1 -- 172.21.15.40:0/744758329 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f416c0ab6c0 con 0x7f4154066690 2024-04-22T05:55:22.978 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 3 to deep-scrub 2024-04-22T05:55:22.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.978+0000 7f4177fcb640 1 -- 172.21.15.40:0/744758329 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4154066690 msgr2=0x7f4154068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.978+0000 7f4177fcb640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4154066690 0x7f4154068b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f416c0ad000 tx=0x7f416c008040 comp rx=0 tx=0).stop 2024-04-22T05:55:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.978+0000 7f4177fcb640 1 -- 172.21.15.40:0/744758329 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f417013e730 msgr2=0x7f417006ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.978+0000 7f4177fcb640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f417013e730 0x7f417006ff60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4168007e30 tx=0x7f416800e4b0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.978+0000 7f4177fcb640 1 -- 172.21.15.40:0/744758329 shutdown_connections 2024-04-22T05:55:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.978+0000 7f4177fcb640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f4154066690 0x7f4154068b50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.978+0000 7f4177fcb640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f417013e730 0x7f417006ff60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.978+0000 7f4177fcb640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41700a79a0 0x7f41701d91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.978+0000 7f4177fcb640 1 --2- 172.21.15.40:0/744758329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f417006df40 0x7f417012e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.978+0000 7f4177fcb640 1 -- 172.21.15.40:0/744758329 >> 172.21.15.40:0/744758329 conn(0x7f41700b3170 msgr2=0x7f417006ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.978+0000 7f4177fcb640 1 -- 172.21.15.40:0/744758329 shutdown_connections 2024-04-22T05:55:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:22.978+0000 7f4177fcb640 1 -- 172.21.15.40:0/744758329 wait complete. 2024-04-22T05:55:22.997 INFO:tasks.ceph:Scrubbing osd.4 2024-04-22T05:55:22.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-22T05:55:23.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.114+0000 7f6a59acb640 1 Processor -- start 2024-04-22T05:55:23.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.114+0000 7f6a59acb640 1 -- start start 2024-04-22T05:55:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a59acb640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6a54141450 0x7f6a54140870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a59acb640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a54140db0 0x7f6a5414bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a59acb640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a54141890 con 0x7f6a54073540 2024-04-22T05:55:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a59acb640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a54141240 con 0x7f6a54141450 2024-04-22T05:55:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a59acb640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a54073920 con 0x7f6a54140db0 2024-04-22T05:55:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a53fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a54140db0 0x7f6a5414bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a53fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a54140db0 0x7f6a5414bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57614/0 (socket says 172.21.15.40:57614) 2024-04-22T05:55:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a53fff640 1 -- 172.21.15.40:0/614853585 learned_addr learned my addr 172.21.15.40:0/614853585 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a53fff640 1 -- 172.21.15.40:0/614853585 >> v2:172.21.15.62:3300/0 conn(0x7f6a54141450 msgr2=0x7f6a54140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a52ffd640 1 --2- 172.21.15.40:0/614853585 >> v2:172.21.15.62:3300/0 conn(0x7f6a54141450 0x7f6a54140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a53fff640 1 --2- 172.21.15.40:0/614853585 >> v2:172.21.15.62:3300/0 conn(0x7f6a54141450 0x7f6a54140870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a53fff640 1 -- 172.21.15.40:0/614853585 >> v1:172.21.15.62:6789/0 conn(0x7f6a54073540 legacy=0x7f6a54145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:23.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a53fff640 1 -- 172.21.15.40:0/614853585 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a54071a70 con 0x7f6a54140db0 2024-04-22T05:55:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a53fff640 1 --2- 172.21.15.40:0/614853585 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a54140db0 0x7f6a5414bd60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009870 tx=0x7f6a4c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33db731a498b1fa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a527fc640 1 -- 172.21.15.40:0/614853585 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0aa5f0 con 0x7f6a54140db0 2024-04-22T05:55:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a527fc640 1 -- 172.21.15.40:0/614853585 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c0aa750 con 0x7f6a54140db0 2024-04-22T05:55:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a527fc640 1 -- 172.21.15.40:0/614853585 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0aaae0 con 0x7f6a54140db0 2024-04-22T05:55:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a59acb640 1 -- 172.21.15.40:0/614853585 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a54140db0 msgr2=0x7f6a5414bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a59acb640 1 --2- 172.21.15.40:0/614853585 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a54140db0 0x7f6a5414bd60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009870 tx=0x7f6a4c01c4d0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a59acb640 1 -- 172.21.15.40:0/614853585 shutdown_connections 2024-04-22T05:55:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a59acb640 1 --2- 172.21.15.40:0/614853585 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a54140db0 0x7f6a5414bd60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a59acb640 1 --2- 172.21.15.40:0/614853585 >> v2:172.21.15.62:3300/0 conn(0x7f6a54141450 0x7f6a54140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a59acb640 1 -- 172.21.15.40:0/614853585 >> 172.21.15.40:0/614853585 conn(0x7f6a540b3fb0 msgr2=0x7f6a540b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:23.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.118+0000 7f6a59acb640 1 -- 172.21.15.40:0/614853585 shutdown_connections 2024-04-22T05:55:23.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a59acb640 1 -- 172.21.15.40:0/614853585 wait complete. 2024-04-22T05:55:23.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a59acb640 1 Processor -- start 2024-04-22T05:55:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a59acb640 1 -- start start 2024-04-22T05:55:23.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a59acb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a54073540 0x7f6a541d8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a59acb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a54140db0 0x7f6a5413fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a59acb640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a54141450 0x7f6a541438e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a59acb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5413f7c0 con 0x7f6a54073540 2024-04-22T05:55:23.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a59acb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a541d93f0 con 0x7f6a54140db0 2024-04-22T05:55:23.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a59acb640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a54136c30 con 0x7f6a54141450 2024-04-22T05:55:23.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a537fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a54073540 0x7f6a541d8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a53fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a54141450 0x7f6a541438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a537fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a54073540 0x7f6a541d8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34164/0 (socket says 172.21.15.40:34164) 2024-04-22T05:55:23.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a53fff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a54141450 0x7f6a541438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57630/0 (socket says 172.21.15.40:57630) 2024-04-22T05:55:23.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a537fe640 1 -- 172.21.15.40:0/3619552776 learned_addr learned my addr 172.21.15.40:0/3619552776 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:23.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a52ffd640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a54140db0 0x7f6a5413fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a53fff640 1 -- 172.21.15.40:0/3619552776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a54140db0 msgr2=0x7f6a5413fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a53fff640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a54140db0 0x7f6a5413fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a53fff640 1 -- 172.21.15.40:0/3619552776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a54073540 msgr2=0x7f6a541d8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a53fff640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a54073540 0x7f6a541d8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a53fff640 1 -- 172.21.15.40:0/3619552776 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c0094d0 con 0x7f6a54141450 2024-04-22T05:55:23.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a537fe640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a54073540 0x7f6a541d8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:23.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.122+0000 7f6a53fff640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a54141450 0x7f6a541438e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6a54073a60 tx=0x7f6a4c01ad10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.126+0000 7f6a4bfff640 1 -- 172.21.15.40:0/3619552776 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0b3800 con 0x7f6a54141450 2024-04-22T05:55:23.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.126+0000 7f6a4bfff640 1 -- 172.21.15.40:0/3619552776 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c0b3980 con 0x7f6a54141450 2024-04-22T05:55:23.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.126+0000 7f6a4bfff640 1 -- 172.21.15.40:0/3619552776 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0b3d30 con 0x7f6a54141450 2024-04-22T05:55:23.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.126+0000 7f6a59acb640 1 -- 172.21.15.40:0/3619552776 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a54140160 con 0x7f6a54141450 2024-04-22T05:55:23.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.126+0000 7f6a59acb640 1 -- 172.21.15.40:0/3619552776 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5413f4a0 con 0x7f6a54141450 2024-04-22T05:55:23.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.126+0000 7f6a59acb640 1 -- 172.21.15.40:0/3619552776 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a541457a0 con 0x7f6a54141450 2024-04-22T05:55:23.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.126+0000 7f6a4bfff640 1 -- 172.21.15.40:0/3619552776 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6a4c0c73a0 con 0x7f6a54141450 2024-04-22T05:55:23.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.126+0000 7f6a4bfff640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6a34066690 0x7f6a34068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.126+0000 7f6a4bfff640 1 -- 172.21.15.40:0/3619552776 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f6a4c0a7070 con 0x7f6a54141450 2024-04-22T05:55:23.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.126+0000 7f6a4bfff640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] conn(0x7f6a3406d7f0 0x7f6a3406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.126+0000 7f6a4bfff640 1 -- 172.21.15.40:0/3619552776 --> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a340702c0 con 0x7f6a3406d7f0 2024-04-22T05:55:23.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.126+0000 7f6a537fe640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6a34066690 0x7f6a34068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.126+0000 7f6a52ffd640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] conn(0x7f6a3406d7f0 0x7f6a3406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.130+0000 7f6a52ffd640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] conn(0x7f6a3406d7f0 0x7f6a3406fc10 crc :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.130+0000 7f6a537fe640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6a34066690 0x7f6a34068b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400c550 tx=0x7f6a4400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.130+0000 7f6a4bfff640 1 -- 172.21.15.40:0/3619552776 <== osd.4 v2:172.21.15.62:6816/25135 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f6a340702c0 con 0x7f6a3406d7f0 2024-04-22T05:55:23.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.134+0000 7f6a4bfff640 1 -- 172.21.15.40:0/3619552776 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a4c0ebe20 con 0x7f6a54141450 2024-04-22T05:55:23.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.142+0000 7f6a59acb640 1 -- 172.21.15.40:0/3619552776 --> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6a5413f940 con 0x7f6a3406d7f0 2024-04-22T05:55:23.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.142+0000 7f6a4bfff640 1 -- 172.21.15.40:0/3619552776 <== osd.4 v2:172.21.15.62:6816/25135 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6a5413f940 con 0x7f6a3406d7f0 2024-04-22T05:55:23.146 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-22T05:55:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T05:55:23.146 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-22T05:55:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.146+0000 7f6a59acb640 1 -- 172.21.15.40:0/3619552776 >> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] conn(0x7f6a3406d7f0 msgr2=0x7f6a3406fc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.146+0000 7f6a59acb640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.62:6816/25135,v1:172.21.15.62:6817/25135] conn(0x7f6a3406d7f0 0x7f6a3406fc10 crc :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.146+0000 7f6a59acb640 1 -- 172.21.15.40:0/3619552776 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6a34066690 msgr2=0x7f6a34068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.146+0000 7f6a59acb640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6a34066690 0x7f6a34068b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400c550 tx=0x7f6a4400b040 comp rx=0 tx=0).stop 2024-04-22T05:55:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.146+0000 7f6a59acb640 1 -- 172.21.15.40:0/3619552776 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a54141450 msgr2=0x7f6a541438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.146+0000 7f6a59acb640 1 --2- 172.21.15.40:0/3619552776 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6a54141450 0x7f6a541438e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6a54073a60 tx=0x7f6a4c01ad10 comp rx=0 tx=0).stop 2024-04-22T05:55:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.146+0000 7f6a53fff640 1 -- 172.21.15.40:0/3619552776 reap_dead start 2024-04-22T05:55:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.146+0000 7f6a59acb640 1 -- 172.21.15.40:0/3619552776 shutdown_connections 2024-04-22T05:55:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.146+0000 7f6a59acb640 1 -- 172.21.15.40:0/3619552776 >> 172.21.15.40:0/3619552776 conn(0x7f6a540b3fb0 msgr2=0x7f6a5406f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.146+0000 7f6a59acb640 1 -- 172.21.15.40:0/3619552776 shutdown_connections 2024-04-22T05:55:23.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.146+0000 7f6a59acb640 1 -- 172.21.15.40:0/3619552776 wait complete. 2024-04-22T05:55:23.162 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-22T05:55:23.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f0bcc3640 1 Processor -- start 2024-04-22T05:55:23.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f0bcc3640 1 -- start start 2024-04-22T05:55:23.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f0bcc3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f0406df40 0x7f9f04141c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f0bcc3640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f04071750 con 0x7f9f0413b030 2024-04-22T05:55:23.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f0bcc3640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f04141750 con 0x7f9f04071370 2024-04-22T05:55:23.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f0bcc3640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040b3ba0 con 0x7f9f0406df40 2024-04-22T05:55:23.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f0925e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f0406df40 0x7f9f04141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f0a260640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9f0413b030 0x7f9f0414bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36826/0 (socket says 172.21.15.40:36826) 2024-04-22T05:55:23.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f0a260640 1 -- 172.21.15.40:0/690713345 learned_addr learned my addr 172.21.15.40:0/690713345 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:55:23.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f0925e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f0406df40 0x7f9f04141c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57632/0 (socket says 172.21.15.40:57632) 2024-04-22T05:55:23.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f0925e640 1 -- 172.21.15.40:0/690713345 >> v1:172.21.15.62:6789/0 conn(0x7f9f04071370 legacy=0x7f9f04070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:23.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f08a5d640 1 -- 172.21.15.40:0/690713345 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809835296 0 0) 0x7f9f04071750 con 0x7f9f0413b030 2024-04-22T05:55:23.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f0925e640 1 -- 172.21.15.40:0/690713345 >> v1:172.21.15.40:6789/0 conn(0x7f9f0413b030 legacy=0x7f9f0414bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.278+0000 7f9f0925e640 1 -- 172.21.15.40:0/690713345 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f041413e0 con 0x7f9f0406df40 2024-04-22T05:55:23.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.282+0000 7f9f0925e640 1 --2- 172.21.15.40:0/690713345 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f0406df40 0x7f9f04141c70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ef40097f0 tx=0x7f9ef401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b424b9efddc65559 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.282+0000 7f9f08a5d640 1 -- 172.21.15.40:0/690713345 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef40ab600 con 0x7f9f0406df40 2024-04-22T05:55:23.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.282+0000 7f9f08a5d640 1 -- 172.21.15.40:0/690713345 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef40a7050 con 0x7f9f0406df40 2024-04-22T05:55:23.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.282+0000 7f9f08a5d640 1 -- 172.21.15.40:0/690713345 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef40ab980 con 0x7f9f0406df40 2024-04-22T05:55:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.282+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/690713345 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f0406df40 msgr2=0x7f9f04141c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.282+0000 7f9f0bcc3640 1 --2- 172.21.15.40:0/690713345 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f0406df40 0x7f9f04141c70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ef40097f0 tx=0x7f9ef401c4c0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.282+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/690713345 shutdown_connections 2024-04-22T05:55:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.282+0000 7f9f0bcc3640 1 --2- 172.21.15.40:0/690713345 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f0406df40 0x7f9f04141c70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.282+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/690713345 >> 172.21.15.40:0/690713345 conn(0x7f9f04073e00 msgr2=0x7f9f040b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.282+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/690713345 shutdown_connections 2024-04-22T05:55:23.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.282+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/690713345 wait complete. 2024-04-22T05:55:23.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.282+0000 7f9f0bcc3640 1 Processor -- start 2024-04-22T05:55:23.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.282+0000 7f9f0bcc3640 1 -- start start 2024-04-22T05:55:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0bcc3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f0406df40 0x7f9f04140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0bcc3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f04071370 0x7f9f0413d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0bcc3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f0413b030 0x7f9f041dd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0bcc3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f041360f0 con 0x7f9f0413b030 2024-04-22T05:55:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0bcc3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f04136390 con 0x7f9f0406df40 2024-04-22T05:55:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0bcc3640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0413cc30 con 0x7f9f04071370 2024-04-22T05:55:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0925e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f04071370 0x7f9f0413d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0925e640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f04071370 0x7f9f0413d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57642/0 (socket says 172.21.15.40:57642) 2024-04-22T05:55:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0925e640 1 -- 172.21.15.40:0/1715686828 learned_addr learned my addr 172.21.15.40:0/1715686828 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f09a5f640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f0406df40 0x7f9f04140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0a260640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f0413b030 0x7f9f041dd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0925e640 1 -- 172.21.15.40:0/1715686828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f0406df40 msgr2=0x7f9f04140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0925e640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f0406df40 0x7f9f04140a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0925e640 1 -- 172.21.15.40:0/1715686828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f0413b030 msgr2=0x7f9f041dd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0925e640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f0413b030 0x7f9f041dd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0925e640 1 -- 172.21.15.40:0/1715686828 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef4009420 con 0x7f9f04071370 2024-04-22T05:55:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0925e640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f04071370 0x7f9f0413d7e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9ef40097f0 tx=0x7f9ef4019260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9efa7fc640 1 -- 172.21.15.40:0/1715686828 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef40abea0 con 0x7f9f04071370 2024-04-22T05:55:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9efa7fc640 1 -- 172.21.15.40:0/1715686828 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef40b4630 con 0x7f9f04071370 2024-04-22T05:55:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9efa7fc640 1 -- 172.21.15.40:0/1715686828 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef40b49e0 con 0x7f9f04071370 2024-04-22T05:55:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/1715686828 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0413deb0 con 0x7f9f04071370 2024-04-22T05:55:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/1715686828 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f041dd910 con 0x7f9f04071370 2024-04-22T05:55:23.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/1715686828 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f0413dd20 con 0x7f9f04071370 2024-04-22T05:55:23.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9efa7fc640 1 -- 172.21.15.40:0/1715686828 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ef40b2040 con 0x7f9f04071370 2024-04-22T05:55:23.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9efa7fc640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9ee8066690 0x7f9ee8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9efa7fc640 1 -- 172.21.15.40:0/1715686828 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f9ef411e1f0 con 0x7f9f04071370 2024-04-22T05:55:23.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.286+0000 7f9f09a5f640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9ee8066690 0x7f9ee8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.290+0000 7f9f09a5f640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9ee8066690 0x7f9ee8068b50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f00000f70 tx=0x7f9f00009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.294+0000 7f9efa7fc640 1 -- 172.21.15.40:0/1715686828 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f9ef40eb630 con 0x7f9f04071370 2024-04-22T05:55:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.482+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/1715686828 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f04136500 con 0x7f9ee8066690 2024-04-22T05:55:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.482+0000 7f9efa7fc640 1 -- 172.21.15.40:0/1715686828 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9f04136500 con 0x7f9ee8066690 2024-04-22T05:55:23.485 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 4 to deep-scrub 2024-04-22T05:55:23.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.486+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/1715686828 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9ee8066690 msgr2=0x7f9ee8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.486+0000 7f9f0bcc3640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9ee8066690 0x7f9ee8068b50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f00000f70 tx=0x7f9f00009040 comp rx=0 tx=0).stop 2024-04-22T05:55:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.486+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/1715686828 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f04071370 msgr2=0x7f9f0413d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.486+0000 7f9f0bcc3640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f04071370 0x7f9f0413d7e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9ef40097f0 tx=0x7f9ef4019260 comp rx=0 tx=0).stop 2024-04-22T05:55:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.486+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/1715686828 shutdown_connections 2024-04-22T05:55:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.486+0000 7f9f0bcc3640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f9ee8066690 0x7f9ee8068b50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.486+0000 7f9f0bcc3640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f0413b030 0x7f9f041dd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.486+0000 7f9f0bcc3640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f9f04071370 0x7f9f0413d7e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.486+0000 7f9f0bcc3640 1 --2- 172.21.15.40:0/1715686828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f0406df40 0x7f9f04140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.486+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/1715686828 >> 172.21.15.40:0/1715686828 conn(0x7f9f04073e00 msgr2=0x7f9f041309b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.486+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/1715686828 shutdown_connections 2024-04-22T05:55:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.486+0000 7f9f0bcc3640 1 -- 172.21.15.40:0/1715686828 wait complete. 2024-04-22T05:55:23.503 INFO:tasks.ceph:Scrubbing osd.5 2024-04-22T05:55:23.503 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-22T05:55:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.626+0000 7f003be91640 1 Processor -- start 2024-04-22T05:55:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.626+0000 7f003be91640 1 -- start start 2024-04-22T05:55:23.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f003be91640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f003413ccf0 0x7f003413d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f003be91640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0034071230 con 0x7f0034070e50 2024-04-22T05:55:23.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f003be91640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003413e670 con 0x7f003406e7d0 2024-04-22T05:55:23.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f003be91640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003413d670 con 0x7f003413ccf0 2024-04-22T05:55:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f003942c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f003413ccf0 0x7f003413d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f0039c2d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0034070e50 0x7f003413dde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36836/0 (socket says 172.21.15.40:36836) 2024-04-22T05:55:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f003942c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f003413ccf0 0x7f003413d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57658/0 (socket says 172.21.15.40:57658) 2024-04-22T05:55:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f0039c2d640 1 -- 172.21.15.40:0/1346152655 learned_addr learned my addr 172.21.15.40:0/1346152655 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:55:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f003942c640 1 -- 172.21.15.40:0/1346152655 >> v1:172.21.15.62:6789/0 conn(0x7f003406e7d0 legacy=0x7f003406ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f003942c640 1 -- 172.21.15.40:0/1346152655 >> v1:172.21.15.40:6789/0 conn(0x7f0034070e50 legacy=0x7f003413dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f0038c2b640 1 -- 172.21.15.40:0/1346152655 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398858497 0 0) 0x7f0034071230 con 0x7f0034070e50 2024-04-22T05:55:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f003942c640 1 -- 172.21.15.40:0/1346152655 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00340a76f0 con 0x7f003413ccf0 2024-04-22T05:55:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f003942c640 1 --2- 172.21.15.40:0/1346152655 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f003413ccf0 0x7f003413d130 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f002400ba80 tx=0x7f002401e830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77da5324dda9bae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.630+0000 7f0038c2b640 1 -- 172.21.15.40:0/1346152655 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00240a8690 con 0x7f003413ccf0 2024-04-22T05:55:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f0038c2b640 1 -- 172.21.15.40:0/1346152655 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00240a87f0 con 0x7f003413ccf0 2024-04-22T05:55:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f0038c2b640 1 -- 172.21.15.40:0/1346152655 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00240a8b80 con 0x7f003413ccf0 2024-04-22T05:55:23.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 -- 172.21.15.40:0/1346152655 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f003413ccf0 msgr2=0x7f003413d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 --2- 172.21.15.40:0/1346152655 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f003413ccf0 0x7f003413d130 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f002400ba80 tx=0x7f002401e830 comp rx=0 tx=0).stop 2024-04-22T05:55:23.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 -- 172.21.15.40:0/1346152655 shutdown_connections 2024-04-22T05:55:23.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 --2- 172.21.15.40:0/1346152655 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f003413ccf0 0x7f003413d130 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 -- 172.21.15.40:0/1346152655 >> 172.21.15.40:0/1346152655 conn(0x7f00340736e0 msgr2=0x7f0034073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:23.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 -- 172.21.15.40:0/1346152655 shutdown_connections 2024-04-22T05:55:23.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 -- 172.21.15.40:0/1346152655 wait complete. 2024-04-22T05:55:23.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 Processor -- start 2024-04-22T05:55:23.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 -- start start 2024-04-22T05:55:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f003406e7d0 0x7f00340ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0034070e50 0x7f0034136fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f003413ccf0 0x7f0034137510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00341327e0 con 0x7f003413ccf0 2024-04-22T05:55:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0034130fa0 con 0x7f0034070e50 2024-04-22T05:55:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00340aeea0 con 0x7f003406e7d0 2024-04-22T05:55:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f0039c2d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f003406e7d0 0x7f00340ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003a42e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f003413ccf0 0x7f0034137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f0039c2d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f003406e7d0 0x7f00340ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57660/0 (socket says 172.21.15.40:57660) 2024-04-22T05:55:23.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f0039c2d640 1 -- 172.21.15.40:0/56908106 learned_addr learned my addr 172.21.15.40:0/56908106 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:23.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003a42e640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f003413ccf0 0x7f0034137510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34190/0 (socket says 172.21.15.40:34190) 2024-04-22T05:55:23.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003942c640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0034070e50 0x7f0034136fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003a42e640 1 -- 172.21.15.40:0/56908106 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f003406e7d0 msgr2=0x7f00340ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003a42e640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f003406e7d0 0x7f00340ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003a42e640 1 -- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0034070e50 msgr2=0x7f0034136fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003a42e640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0034070e50 0x7f0034136fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003a42e640 1 -- 172.21.15.40:0/56908106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002400b6b0 con 0x7f003413ccf0 2024-04-22T05:55:23.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003942c640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0034070e50 0x7f0034136fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:55:23.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f0039c2d640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f003406e7d0 0x7f00340ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:23.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003a42e640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f003413ccf0 0x7f0034137510 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f002c00c700 tx=0x7f002c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f002a7fc640 1 -- 172.21.15.40:0/56908106 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c003b80 con 0x7f003413ccf0 2024-04-22T05:55:23.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f002a7fc640 1 -- 172.21.15.40:0/56908106 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c003ce0 con 0x7f003413ccf0 2024-04-22T05:55:23.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 -- 172.21.15.40:0/56908106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0034132e80 con 0x7f003413ccf0 2024-04-22T05:55:23.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f002a7fc640 1 -- 172.21.15.40:0/56908106 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c00aa30 con 0x7f003413ccf0 2024-04-22T05:55:23.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.634+0000 7f003be91640 1 -- 172.21.15.40:0/56908106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0034131480 con 0x7f003413ccf0 2024-04-22T05:55:23.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.638+0000 7f003be91640 1 -- 172.21.15.40:0/56908106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f003404e9c0 con 0x7f003413ccf0 2024-04-22T05:55:23.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.638+0000 7f002a7fc640 1 -- 172.21.15.40:0/56908106 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f002c00ab90 con 0x7f003413ccf0 2024-04-22T05:55:23.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.638+0000 7f002a7fc640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f00180666c0 0x7f0018068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.638+0000 7f002a7fc640 1 -- 172.21.15.40:0/56908106 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f002c083ee0 con 0x7f003413ccf0 2024-04-22T05:55:23.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.638+0000 7f002a7fc640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] conn(0x7f001806d820 0x7f001806fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.638+0000 7f002a7fc640 1 -- 172.21.15.40:0/56908106 --> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f00180702f0 con 0x7f001806d820 2024-04-22T05:55:23.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.638+0000 7f002a7fc640 1 -- 172.21.15.40:0/56908106 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f002c0865a0 con 0x7f003413ccf0 2024-04-22T05:55:23.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.638+0000 7f0039c2d640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f00180666c0 0x7f0018068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.638+0000 7f003942c640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] conn(0x7f001806d820 0x7f001806fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.638+0000 7f0039c2d640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f00180666c0 0x7f0018068b80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0030007b20 tx=0x7f0030007a70 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.638+0000 7f003942c640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] conn(0x7f001806d820 0x7f001806fc40 crc :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.642+0000 7f002a7fc640 1 -- 172.21.15.40:0/56908106 <== osd.5 v2:172.21.15.62:6824/25137 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f00180702f0 con 0x7f001806d820 2024-04-22T05:55:23.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.654+0000 7f003be91640 1 -- 172.21.15.40:0/56908106 --> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0034131110 con 0x7f001806d820 2024-04-22T05:55:23.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.654+0000 7f002a7fc640 1 -- 172.21.15.40:0/56908106 <== osd.5 v2:172.21.15.62:6824/25137 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0034131110 con 0x7f001806d820 2024-04-22T05:55:23.656 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-22T05:55:23.656 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T05:55:23.656 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-22T05:55:23.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.654+0000 7f003be91640 1 -- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] conn(0x7f001806d820 msgr2=0x7f001806fc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.654+0000 7f003be91640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:6824/25137,v1:172.21.15.62:6825/25137] conn(0x7f001806d820 0x7f001806fc40 crc :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.654+0000 7f003be91640 1 -- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f00180666c0 msgr2=0x7f0018068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.654+0000 7f003be91640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f00180666c0 0x7f0018068b80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0030007b20 tx=0x7f0030007a70 comp rx=0 tx=0).stop 2024-04-22T05:55:23.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.654+0000 7f003be91640 1 -- 172.21.15.40:0/56908106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f003413ccf0 msgr2=0x7f0034137510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.654+0000 7f003be91640 1 --2- 172.21.15.40:0/56908106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f003413ccf0 0x7f0034137510 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f002c00c700 tx=0x7f002c00cbc0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.654+0000 7f003a42e640 1 -- 172.21.15.40:0/56908106 reap_dead start 2024-04-22T05:55:23.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.654+0000 7f003be91640 1 -- 172.21.15.40:0/56908106 shutdown_connections 2024-04-22T05:55:23.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.654+0000 7f003be91640 1 -- 172.21.15.40:0/56908106 >> 172.21.15.40:0/56908106 conn(0x7f00340736e0 msgr2=0x7f003413a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:23.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.654+0000 7f003be91640 1 -- 172.21.15.40:0/56908106 shutdown_connections 2024-04-22T05:55:23.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.654+0000 7f003be91640 1 -- 172.21.15.40:0/56908106 wait complete. 2024-04-22T05:55:23.672 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-22T05:55:23.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.778+0000 7f6d476aa640 1 Processor -- start 2024-04-22T05:55:23.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.778+0000 7f6d476aa640 1 -- start start 2024-04-22T05:55:23.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.778+0000 7f6d476aa640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6d400a7450 0x7f6d4013a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.778+0000 7f6d476aa640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d400a8710 con 0x7f6d40070ba0 2024-04-22T05:55:23.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.778+0000 7f6d476aa640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d400af6d0 con 0x7f6d40070f80 2024-04-22T05:55:23.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.778+0000 7f6d476aa640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d400a8530 con 0x7f6d400a7450 2024-04-22T05:55:23.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.778+0000 7f6d45c47640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6d400a7450 0x7f6d4013a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.778+0000 7f6d45c47640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6d400a7450 0x7f6d4013a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34204/0 (socket says 172.21.15.40:34204) 2024-04-22T05:55:23.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.778+0000 7f6d45c47640 1 -- 172.21.15.40:0/1723233823 learned_addr learned my addr 172.21.15.40:0/1723233823 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:23.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.778+0000 7f6d45c47640 1 -- 172.21.15.40:0/1723233823 >> v1:172.21.15.62:6789/0 conn(0x7f6d40070f80 legacy=0x7f6d4013e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:23.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.778+0000 7f6d45c47640 1 -- 172.21.15.40:0/1723233823 >> v1:172.21.15.40:6789/0 conn(0x7f6d40070ba0 legacy=0x7f6d4006e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.778+0000 7f6d45c47640 1 -- 172.21.15.40:0/1723233823 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d400af860 con 0x7f6d400a7450 2024-04-22T05:55:23.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.782+0000 7f6d45c47640 1 --2- 172.21.15.40:0/1723233823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d400a7450 0x7f6d4013a810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800bb10 tx=0x7f6d3801eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660fac4805b255bf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.782+0000 7f6d37fff640 1 -- 172.21.15.40:0/1723233823 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d380a7aa0 con 0x7f6d400a7450 2024-04-22T05:55:23.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.782+0000 7f6d37fff640 1 -- 172.21.15.40:0/1723233823 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d380a7c00 con 0x7f6d400a7450 2024-04-22T05:55:23.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.782+0000 7f6d37fff640 1 -- 172.21.15.40:0/1723233823 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d380af440 con 0x7f6d400a7450 2024-04-22T05:55:23.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.782+0000 7f6d476aa640 1 -- 172.21.15.40:0/1723233823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d400a7450 msgr2=0x7f6d4013a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.782+0000 7f6d476aa640 1 --2- 172.21.15.40:0/1723233823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d400a7450 0x7f6d4013a810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800bb10 tx=0x7f6d3801eb40 comp rx=0 tx=0).stop 2024-04-22T05:55:23.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.782+0000 7f6d476aa640 1 -- 172.21.15.40:0/1723233823 shutdown_connections 2024-04-22T05:55:23.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.782+0000 7f6d476aa640 1 --2- 172.21.15.40:0/1723233823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d400a7450 0x7f6d4013a810 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.782+0000 7f6d476aa640 1 -- 172.21.15.40:0/1723233823 >> 172.21.15.40:0/1723233823 conn(0x7f6d400b3170 msgr2=0x7f6d400b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:23.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.782+0000 7f6d476aa640 1 -- 172.21.15.40:0/1723233823 shutdown_connections 2024-04-22T05:55:23.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.782+0000 7f6d476aa640 1 -- 172.21.15.40:0/1723233823 wait complete. 2024-04-22T05:55:23.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d476aa640 1 Processor -- start 2024-04-22T05:55:23.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d476aa640 1 -- start start 2024-04-22T05:55:23.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d476aa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d40070ba0 0x7f6d4012b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d476aa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d40070f80 0x7f6d4012c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d476aa640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6d400a7450 0x7f6d4013b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d476aa640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d40071440 con 0x7f6d40070f80 2024-04-22T05:55:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d476aa640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4012cb20 con 0x7f6d40070ba0 2024-04-22T05:55:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d476aa640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4012be70 con 0x7f6d400a7450 2024-04-22T05:55:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d44c45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d40070f80 0x7f6d4012c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d45c47640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6d400a7450 0x7f6d4013b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d44c45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d40070f80 0x7f6d4012c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34220/0 (socket says 172.21.15.40:34220) 2024-04-22T05:55:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d44c45640 1 -- 172.21.15.40:0/1002486932 learned_addr learned my addr 172.21.15.40:0/1002486932 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d45446640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d40070ba0 0x7f6d4012b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d45c47640 1 -- 172.21.15.40:0/1002486932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d40070ba0 msgr2=0x7f6d4012b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d45c47640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d40070ba0 0x7f6d4012b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d45c47640 1 -- 172.21.15.40:0/1002486932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d40070f80 msgr2=0x7f6d4012c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d45c47640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d40070f80 0x7f6d4012c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d45c47640 1 -- 172.21.15.40:0/1002486932 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3800b770 con 0x7f6d400a7450 2024-04-22T05:55:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d45446640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d40070ba0 0x7f6d4012b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:55:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d44c45640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d40070f80 0x7f6d4012c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d45c47640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6d400a7450 0x7f6d4013b6a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6d380ae000 tx=0x7f6d3801b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d35ffb640 1 -- 172.21.15.40:0/1002486932 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d380b8600 con 0x7f6d400a7450 2024-04-22T05:55:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d35ffb640 1 -- 172.21.15.40:0/1002486932 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d380b8760 con 0x7f6d400a7450 2024-04-22T05:55:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d476aa640 1 -- 172.21.15.40:0/1002486932 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4012c150 con 0x7f6d400a7450 2024-04-22T05:55:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d35ffb640 1 -- 172.21.15.40:0/1002486932 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d380b8af0 con 0x7f6d400a7450 2024-04-22T05:55:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d476aa640 1 -- 172.21.15.40:0/1002486932 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4013d2f0 con 0x7f6d400a7450 2024-04-22T05:55:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d476aa640 1 -- 172.21.15.40:0/1002486932 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4013d2f0 con 0x7f6d400a7450 2024-04-22T05:55:23.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.786+0000 7f6d35ffb640 1 -- 172.21.15.40:0/1002486932 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6d380b6040 con 0x7f6d400a7450 2024-04-22T05:55:23.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.790+0000 7f6d35ffb640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6d24066690 0x7f6d24068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:23.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.790+0000 7f6d35ffb640 1 -- 172.21.15.40:0/1002486932 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f6d38121180 con 0x7f6d400a7450 2024-04-22T05:55:23.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.790+0000 7f6d45446640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6d24066690 0x7f6d24068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:23.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.790+0000 7f6d45446640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6d24066690 0x7f6d24068b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c007fd0 tx=0x7f6d3c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.794+0000 7f6d35ffb640 1 -- 172.21.15.40:0/1002486932 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f6d380ee530 con 0x7f6d400a7450 2024-04-22T05:55:23.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.946+0000 7f6d476aa640 1 -- 172.21.15.40:0/1002486932 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d4012bfb0 con 0x7f6d24066690 2024-04-22T05:55:23.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.946+0000 7f6d35ffb640 1 -- 172.21.15.40:0/1002486932 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6d3c006b60 con 0x7f6d24066690 2024-04-22T05:55:23.949 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 5 to deep-scrub 2024-04-22T05:55:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.950+0000 7f6d476aa640 1 -- 172.21.15.40:0/1002486932 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6d24066690 msgr2=0x7f6d24068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.950+0000 7f6d476aa640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6d24066690 0x7f6d24068b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c007fd0 tx=0x7f6d3c009040 comp rx=0 tx=0).stop 2024-04-22T05:55:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.950+0000 7f6d476aa640 1 -- 172.21.15.40:0/1002486932 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6d400a7450 msgr2=0x7f6d4013b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.950+0000 7f6d476aa640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6d400a7450 0x7f6d4013b6a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6d380ae000 tx=0x7f6d3801b8e0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.950+0000 7f6d476aa640 1 -- 172.21.15.40:0/1002486932 shutdown_connections 2024-04-22T05:55:23.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.950+0000 7f6d476aa640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6d24066690 0x7f6d24068b50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.950+0000 7f6d476aa640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6d400a7450 0x7f6d4013b6a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.950+0000 7f6d476aa640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d40070f80 0x7f6d4012c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.950+0000 7f6d476aa640 1 --2- 172.21.15.40:0/1002486932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d40070ba0 0x7f6d4012b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:23.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.950+0000 7f6d476aa640 1 -- 172.21.15.40:0/1002486932 >> 172.21.15.40:0/1002486932 conn(0x7f6d400b3170 msgr2=0x7f6d4006e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:23.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.950+0000 7f6d476aa640 1 -- 172.21.15.40:0/1002486932 shutdown_connections 2024-04-22T05:55:23.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:23.950+0000 7f6d476aa640 1 -- 172.21.15.40:0/1002486932 wait complete. 2024-04-22T05:55:23.967 INFO:tasks.ceph:Scrubbing osd.6 2024-04-22T05:55:23.967 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-22T05:55:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.090+0000 7f6decdc1640 1 Processor -- start 2024-04-22T05:55:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.090+0000 7f6decdc1640 1 -- start start 2024-04-22T05:55:24.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6decdc1640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6de8142970 0x7f6de8142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6decdc1640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6de8143340 0x7f6de8149b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6decdc1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de80aa120 con 0x7f6de8071480 2024-04-22T05:55:24.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6decdc1640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de8071860 con 0x7f6de8142970 2024-04-22T05:55:24.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6decdc1640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de80a8060 con 0x7f6de8143340 2024-04-22T05:55:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6de6d9d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6de8143340 0x7f6de8149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6de6d9d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6de8143340 0x7f6de8149b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57672/0 (socket says 172.21.15.40:57672) 2024-04-22T05:55:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6de6d9d640 1 -- 172.21.15.40:0/3511873581 learned_addr learned my addr 172.21.15.40:0/3511873581 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6de5d9b640 1 --2- 172.21.15.40:0/3511873581 >> v2:172.21.15.62:3300/0 conn(0x7f6de8142970 0x7f6de8142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6de6d9d640 1 -- 172.21.15.40:0/3511873581 >> v2:172.21.15.62:3300/0 conn(0x7f6de8142970 msgr2=0x7f6de8142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6de6d9d640 1 --2- 172.21.15.40:0/3511873581 >> v2:172.21.15.62:3300/0 conn(0x7f6de8142970 0x7f6de8142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6de6d9d640 1 -- 172.21.15.40:0/3511873581 >> v1:172.21.15.62:6789/0 conn(0x7f6de8071480 legacy=0x7f6de8138e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6de6d9d640 1 -- 172.21.15.40:0/3511873581 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de80b0850 con 0x7f6de8143340 2024-04-22T05:55:24.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6de5d9b640 1 --2- 172.21.15.40:0/3511873581 >> v2:172.21.15.62:3300/0 conn(0x7f6de8142970 0x7f6de8142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:55:24.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6de6d9d640 1 --2- 172.21.15.40:0/3511873581 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6de8143340 0x7f6de8149b70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009e50 tx=0x7f6de001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e46aabf11cd5c319 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6de559a640 1 -- 172.21.15.40:0/3511873581 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de00a56d0 con 0x7f6de8143340 2024-04-22T05:55:24.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6de559a640 1 -- 172.21.15.40:0/3511873581 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de00a5830 con 0x7f6de8143340 2024-04-22T05:55:24.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.094+0000 7f6de559a640 1 -- 172.21.15.40:0/3511873581 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de00a5bc0 con 0x7f6de8143340 2024-04-22T05:55:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 -- 172.21.15.40:0/3511873581 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6de8143340 msgr2=0x7f6de8149b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 --2- 172.21.15.40:0/3511873581 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6de8143340 0x7f6de8149b70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009e50 tx=0x7f6de001c940 comp rx=0 tx=0).stop 2024-04-22T05:55:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 -- 172.21.15.40:0/3511873581 shutdown_connections 2024-04-22T05:55:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 --2- 172.21.15.40:0/3511873581 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6de8143340 0x7f6de8149b70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 --2- 172.21.15.40:0/3511873581 >> v2:172.21.15.62:3300/0 conn(0x7f6de8142970 0x7f6de8142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 -- 172.21.15.40:0/3511873581 >> 172.21.15.40:0/3511873581 conn(0x7f6de80b3fb0 msgr2=0x7f6de80b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 -- 172.21.15.40:0/3511873581 shutdown_connections 2024-04-22T05:55:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 -- 172.21.15.40:0/3511873581 wait complete. 2024-04-22T05:55:24.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 Processor -- start 2024-04-22T05:55:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 -- start start 2024-04-22T05:55:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de8071480 0x7f6de8142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de8142970 0x7f6de80ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6de8143340 0x7f6de80aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de8138030 con 0x7f6de8142970 2024-04-22T05:55:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de8136df0 con 0x7f6de8071480 2024-04-22T05:55:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de8137090 con 0x7f6de8143340 2024-04-22T05:55:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6de5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de8142970 0x7f6de80ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6de6d9d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6de8143340 0x7f6de80aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6de5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de8142970 0x7f6de80ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34224/0 (socket says 172.21.15.40:34224) 2024-04-22T05:55:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6de5d9b640 1 -- 172.21.15.40:0/886726394 learned_addr learned my addr 172.21.15.40:0/886726394 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6de659c640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de8071480 0x7f6de8142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6de5d9b640 1 -- 172.21.15.40:0/886726394 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6de8143340 msgr2=0x7f6de80aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6de5d9b640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6de8143340 0x7f6de80aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6de5d9b640 1 -- 172.21.15.40:0/886726394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de8071480 msgr2=0x7f6de8142180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6de5d9b640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de8071480 0x7f6de8142180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6de5d9b640 1 -- 172.21.15.40:0/886726394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de0009b30 con 0x7f6de8142970 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6de6d9d640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f6de8143340 0x7f6de80aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6de5d9b640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de8142970 0x7f6de80ae9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c7c0 tx=0x7f6dd800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6dd6ffd640 1 -- 172.21.15.40:0/886726394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8007a30 con 0x7f6de8142970 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6dd6ffd640 1 -- 172.21.15.40:0/886726394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd8007b90 con 0x7f6de8142970 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6dd6ffd640 1 -- 172.21.15.40:0/886726394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800ce70 con 0x7f6de8142970 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 -- 172.21.15.40:0/886726394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de8137bd0 con 0x7f6de8142970 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.098+0000 7f6decdc1640 1 -- 172.21.15.40:0/886726394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de80aff10 con 0x7f6de8142970 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.102+0000 7f6decdc1640 1 -- 172.21.15.40:0/886726394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6de814c8d0 con 0x7f6de8142970 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.102+0000 7f6dd6ffd640 1 -- 172.21.15.40:0/886726394 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6dd80113f0 con 0x7f6de8142970 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.102+0000 7f6dd6ffd640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6dc8066690 0x7f6dc8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.102+0000 7f6dd6ffd640 1 -- 172.21.15.40:0/886726394 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f6dd8084790 con 0x7f6de8142970 2024-04-22T05:55:24.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.102+0000 7f6dd6ffd640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] conn(0x7f6dc806d7f0 0x7f6dc806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.102+0000 7f6dd6ffd640 1 -- 172.21.15.40:0/886726394 --> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6dc80702c0 con 0x7f6dc806d7f0 2024-04-22T05:55:24.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.102+0000 7f6dd6ffd640 1 -- 172.21.15.40:0/886726394 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dd8084be0 con 0x7f6de8142970 2024-04-22T05:55:24.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.102+0000 7f6de659c640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6dc8066690 0x7f6dc8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.102+0000 7f6de6d9d640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] conn(0x7f6dc806d7f0 0x7f6dc806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.102+0000 7f6de659c640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6dc8066690 0x7f6dc8068b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00b810 tx=0x7f6ddc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.102+0000 7f6de6d9d640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] conn(0x7f6dc806d7f0 0x7f6dc806fc10 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.106+0000 7f6dd6ffd640 1 -- 172.21.15.40:0/886726394 <== osd.6 v2:172.21.15.62:6800/25136 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f6dc80702c0 con 0x7f6dc806d7f0 2024-04-22T05:55:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.118+0000 7f6decdc1640 1 -- 172.21.15.40:0/886726394 --> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6de804e9c0 con 0x7f6dc806d7f0 2024-04-22T05:55:24.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.118+0000 7f6dd6ffd640 1 -- 172.21.15.40:0/886726394 <== osd.6 v2:172.21.15.62:6800/25136 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6de804e9c0 con 0x7f6dc806d7f0 2024-04-22T05:55:24.120 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-22T05:55:24.120 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T05:55:24.121 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-22T05:55:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.118+0000 7f6decdc1640 1 -- 172.21.15.40:0/886726394 >> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] conn(0x7f6dc806d7f0 msgr2=0x7f6dc806fc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.118+0000 7f6decdc1640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.62:6800/25136,v1:172.21.15.62:6801/25136] conn(0x7f6dc806d7f0 0x7f6dc806fc10 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.118+0000 7f6decdc1640 1 -- 172.21.15.40:0/886726394 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6dc8066690 msgr2=0x7f6dc8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.118+0000 7f6decdc1640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f6dc8066690 0x7f6dc8068b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00b810 tx=0x7f6ddc008040 comp rx=0 tx=0).stop 2024-04-22T05:55:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.118+0000 7f6decdc1640 1 -- 172.21.15.40:0/886726394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de8142970 msgr2=0x7f6de80ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.118+0000 7f6decdc1640 1 --2- 172.21.15.40:0/886726394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de8142970 0x7f6de80ae9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c7c0 tx=0x7f6dd800cc80 comp rx=0 tx=0).stop 2024-04-22T05:55:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.118+0000 7f6de6d9d640 1 -- 172.21.15.40:0/886726394 reap_dead start 2024-04-22T05:55:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.118+0000 7f6decdc1640 1 -- 172.21.15.40:0/886726394 shutdown_connections 2024-04-22T05:55:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.118+0000 7f6decdc1640 1 -- 172.21.15.40:0/886726394 >> 172.21.15.40:0/886726394 conn(0x7f6de80b3fb0 msgr2=0x7f6de80b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.118+0000 7f6decdc1640 1 -- 172.21.15.40:0/886726394 shutdown_connections 2024-04-22T05:55:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.118+0000 7f6decdc1640 1 -- 172.21.15.40:0/886726394 wait complete. 2024-04-22T05:55:24.136 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-22T05:55:24.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e39e6640 1 Processor -- start 2024-04-22T05:55:24.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e39e6640 1 -- start start 2024-04-22T05:55:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e39e6640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd3dc13e410 0x7fd3dc13e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e39e6640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc06d2c0 0x7fd3dc06d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e39e6640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3dc0b46b0 con 0x7fd3dc0b42d0 2024-04-22T05:55:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e39e6640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3dc13ede0 con 0x7fd3dc13e410 2024-04-22T05:55:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e39e6640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc0ac2a0 con 0x7fd3dc06d2c0 2024-04-22T05:55:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e1f83640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc06d2c0 0x7fd3dc06d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e1f83640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc06d2c0 0x7fd3dc06d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57690/0 (socket says 172.21.15.40:57690) 2024-04-22T05:55:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e1f83640 1 -- 172.21.15.40:0/902692022 learned_addr learned my addr 172.21.15.40:0/902692022 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e0f81640 1 --2- 172.21.15.40:0/902692022 >> v2:172.21.15.62:3300/0 conn(0x7fd3dc13e410 0x7fd3dc13e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e1f83640 1 -- 172.21.15.40:0/902692022 >> v2:172.21.15.62:3300/0 conn(0x7fd3dc13e410 msgr2=0x7fd3dc13e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e1f83640 1 --2- 172.21.15.40:0/902692022 >> v2:172.21.15.62:3300/0 conn(0x7fd3dc13e410 0x7fd3dc13e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e1f83640 1 -- 172.21.15.40:0/902692022 >> v1:172.21.15.62:6789/0 conn(0x7fd3dc0b42d0 legacy=0x7fd3dc141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:24.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.246+0000 7fd3e1f83640 1 -- 172.21.15.40:0/902692022 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc0b5fb0 con 0x7fd3dc06d2c0 2024-04-22T05:55:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e1f83640 1 --2- 172.21.15.40:0/902692022 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc06d2c0 0x7fd3dc06d730 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009e50 tx=0x7fd3d401c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a419606e6bb0df4f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3d3fff640 1 -- 172.21.15.40:0/902692022 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d40a5720 con 0x7fd3dc06d2c0 2024-04-22T05:55:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3d3fff640 1 -- 172.21.15.40:0/902692022 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d40a5880 con 0x7fd3dc06d2c0 2024-04-22T05:55:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3d3fff640 1 -- 172.21.15.40:0/902692022 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d40a5c10 con 0x7fd3dc06d2c0 2024-04-22T05:55:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 -- 172.21.15.40:0/902692022 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc06d2c0 msgr2=0x7fd3dc06d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 --2- 172.21.15.40:0/902692022 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc06d2c0 0x7fd3dc06d730 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009e50 tx=0x7fd3d401c880 comp rx=0 tx=0).stop 2024-04-22T05:55:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 -- 172.21.15.40:0/902692022 shutdown_connections 2024-04-22T05:55:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 --2- 172.21.15.40:0/902692022 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc06d2c0 0x7fd3dc06d730 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 --2- 172.21.15.40:0/902692022 >> v2:172.21.15.62:3300/0 conn(0x7fd3dc13e410 0x7fd3dc13e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 -- 172.21.15.40:0/902692022 >> 172.21.15.40:0/902692022 conn(0x7fd3dc0b1c60 msgr2=0x7fd3dc0b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 -- 172.21.15.40:0/902692022 shutdown_connections 2024-04-22T05:55:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 -- 172.21.15.40:0/902692022 wait complete. 2024-04-22T05:55:24.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 Processor -- start 2024-04-22T05:55:24.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 -- start start 2024-04-22T05:55:24.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc06d2c0 0x7fd3dc135560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc0b42d0 0x7fd3dc135ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3dc13e410 0x7fd3dc1361e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc137ea0 con 0x7fd3dc13e410 2024-04-22T05:55:24.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc136720 con 0x7fd3dc06d2c0 2024-04-22T05:55:24.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc138140 con 0x7fd3dc0b42d0 2024-04-22T05:55:24.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e0f81640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc0b42d0 0x7fd3dc135ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e0f81640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc0b42d0 0x7fd3dc135ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57706/0 (socket says 172.21.15.40:57706) 2024-04-22T05:55:24.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e0f81640 1 -- 172.21.15.40:0/1866101008 learned_addr learned my addr 172.21.15.40:0/1866101008 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:24.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e1f83640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3dc13e410 0x7fd3dc1361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e1782640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc06d2c0 0x7fd3dc135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e0f81640 1 -- 172.21.15.40:0/1866101008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc06d2c0 msgr2=0x7fd3dc135560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e0f81640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc06d2c0 0x7fd3dc135560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e0f81640 1 -- 172.21.15.40:0/1866101008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3dc13e410 msgr2=0x7fd3dc1361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e0f81640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3dc13e410 0x7fd3dc1361e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e0f81640 1 -- 172.21.15.40:0/1866101008 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4009b30 con 0x7fd3dc0b42d0 2024-04-22T05:55:24.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e0f81640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc0b42d0 0x7fd3dc135ca0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00d5f0 tx=0x7fd3cc00dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3d1ffb640 1 -- 172.21.15.40:0/1866101008 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc0125f0 con 0x7fd3dc0b42d0 2024-04-22T05:55:24.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3d1ffb640 1 -- 172.21.15.40:0/1866101008 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc012750 con 0x7fd3dc0b42d0 2024-04-22T05:55:24.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 -- 172.21.15.40:0/1866101008 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3dc137170 con 0x7fd3dc0b42d0 2024-04-22T05:55:24.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3d1ffb640 1 -- 172.21.15.40:0/1866101008 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc012ae0 con 0x7fd3dc0b42d0 2024-04-22T05:55:24.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.250+0000 7fd3e39e6640 1 -- 172.21.15.40:0/1866101008 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3dc1347e0 con 0x7fd3dc0b42d0 2024-04-22T05:55:24.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.254+0000 7fd3e39e6640 1 -- 172.21.15.40:0/1866101008 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3dc04e690 con 0x7fd3dc0b42d0 2024-04-22T05:55:24.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.254+0000 7fd3d1ffb640 1 -- 172.21.15.40:0/1866101008 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd3cc012c40 con 0x7fd3dc0b42d0 2024-04-22T05:55:24.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.254+0000 7fd3d1ffb640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fd3c0066690 0x7fd3c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.254+0000 7fd3d1ffb640 1 -- 172.21.15.40:0/1866101008 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7fd3cc084480 con 0x7fd3dc0b42d0 2024-04-22T05:55:24.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.254+0000 7fd3e1782640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fd3c0066690 0x7fd3c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.254+0000 7fd3e1782640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fd3c0066690 0x7fd3c0068b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8004e30 tx=0x7fd3d8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.258+0000 7fd3d1ffb640 1 -- 172.21.15.40:0/1866101008 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7fd3cc0518c0 con 0x7fd3dc0b42d0 2024-04-22T05:55:24.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.410+0000 7fd3e39e6640 1 -- 172.21.15.40:0/1866101008 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3dc14bb50 con 0x7fd3c0066690 2024-04-22T05:55:24.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.414+0000 7fd3d1ffb640 1 -- 172.21.15.40:0/1866101008 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd3dc14bb50 con 0x7fd3c0066690 2024-04-22T05:55:24.414 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 6 to deep-scrub 2024-04-22T05:55:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.414+0000 7fd3e39e6640 1 -- 172.21.15.40:0/1866101008 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fd3c0066690 msgr2=0x7fd3c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.414+0000 7fd3e39e6640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fd3c0066690 0x7fd3c0068b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8004e30 tx=0x7fd3d8009040 comp rx=0 tx=0).stop 2024-04-22T05:55:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.414+0000 7fd3e39e6640 1 -- 172.21.15.40:0/1866101008 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc0b42d0 msgr2=0x7fd3dc135ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.414+0000 7fd3e39e6640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc0b42d0 0x7fd3dc135ca0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00d5f0 tx=0x7fd3cc00dab0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.414+0000 7fd3e39e6640 1 -- 172.21.15.40:0/1866101008 shutdown_connections 2024-04-22T05:55:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.414+0000 7fd3e39e6640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fd3c0066690 0x7fd3c0068b50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.414+0000 7fd3e39e6640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3dc13e410 0x7fd3dc1361e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.414+0000 7fd3e39e6640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fd3dc0b42d0 0x7fd3dc135ca0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.414+0000 7fd3e39e6640 1 --2- 172.21.15.40:0/1866101008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc06d2c0 0x7fd3dc135560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.414+0000 7fd3e39e6640 1 -- 172.21.15.40:0/1866101008 >> 172.21.15.40:0/1866101008 conn(0x7fd3dc0b1c60 msgr2=0x7fd3dc13d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.414+0000 7fd3e39e6640 1 -- 172.21.15.40:0/1866101008 shutdown_connections 2024-04-22T05:55:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.414+0000 7fd3e39e6640 1 -- 172.21.15.40:0/1866101008 wait complete. 2024-04-22T05:55:24.434 INFO:tasks.ceph:Scrubbing osd.7 2024-04-22T05:55:24.434 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-22T05:55:24.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3ee07d640 1 Processor -- start 2024-04-22T05:55:24.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3ee07d640 1 -- start start 2024-04-22T05:55:24.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3ee07d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa3e80a7ab0 0x7fa3e80a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3ee07d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa3e813fe20 0x7fa3e806df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3ee07d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e813f900 con 0x7fa3e806e480 2024-04-22T05:55:24.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3ee07d640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e80a8d00 con 0x7fa3e80a7ab0 2024-04-22T05:55:24.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3ee07d640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80a8f80 con 0x7fa3e813fe20 2024-04-22T05:55:24.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3dffff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa3e813fe20 0x7fa3e806df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3e7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa3e806e480 0x7fa3e813c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36850/0 (socket says 172.21.15.40:36850) 2024-04-22T05:55:24.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3e7fff640 1 -- 172.21.15.40:0/2338522419 learned_addr learned my addr 172.21.15.40:0/2338522419 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:55:24.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3e77fe640 1 --2- 172.21.15.40:0/2338522419 >> v2:172.21.15.62:3300/0 conn(0x7fa3e80a7ab0 0x7fa3e80a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3e6ffd640 1 -- 172.21.15.40:0/2338522419 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390794272 0 0) 0x7fa3e813f900 con 0x7fa3e806e480 2024-04-22T05:55:24.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3e6ffd640 1 -- 172.21.15.40:0/2338522419 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3d4003530 con 0x7fa3e806e480 2024-04-22T05:55:24.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3dffff640 1 -- 172.21.15.40:0/2338522419 >> v2:172.21.15.62:3300/0 conn(0x7fa3e80a7ab0 msgr2=0x7fa3e80a7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:24.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3dffff640 1 --2- 172.21.15.40:0/2338522419 >> v2:172.21.15.62:3300/0 conn(0x7fa3e80a7ab0 0x7fa3e80a7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3dffff640 1 -- 172.21.15.40:0/2338522419 >> v1:172.21.15.40:6789/0 conn(0x7fa3e806e480 legacy=0x7fa3e813c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.562+0000 7fa3dffff640 1 -- 172.21.15.40:0/2338522419 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e8140280 con 0x7fa3e813fe20 2024-04-22T05:55:24.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3dffff640 1 --2- 172.21.15.40:0/2338522419 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa3e813fe20 0x7fa3e806df40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00097c0 tx=0x7fa3d001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ba629a1855711ca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3e6ffd640 1 -- 172.21.15.40:0/2338522419 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00ad640 con 0x7fa3e813fe20 2024-04-22T05:55:24.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3e6ffd640 1 -- 172.21.15.40:0/2338522419 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d00ad7a0 con 0x7fa3e813fe20 2024-04-22T05:55:24.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3e6ffd640 1 -- 172.21.15.40:0/2338522419 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00adb30 con 0x7fa3e813fe20 2024-04-22T05:55:24.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3ee07d640 1 -- 172.21.15.40:0/2338522419 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa3e813fe20 msgr2=0x7fa3e806df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3ee07d640 1 --2- 172.21.15.40:0/2338522419 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa3e813fe20 0x7fa3e806df40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00097c0 tx=0x7fa3d001c480 comp rx=0 tx=0).stop 2024-04-22T05:55:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3ee07d640 1 -- 172.21.15.40:0/2338522419 shutdown_connections 2024-04-22T05:55:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3ee07d640 1 --2- 172.21.15.40:0/2338522419 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa3e813fe20 0x7fa3e806df40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3ee07d640 1 --2- 172.21.15.40:0/2338522419 >> v2:172.21.15.62:3300/0 conn(0x7fa3e80a7ab0 0x7fa3e80a7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3ee07d640 1 -- 172.21.15.40:0/2338522419 >> 172.21.15.40:0/2338522419 conn(0x7fa3e80736e0 msgr2=0x7fa3e8073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3ee07d640 1 -- 172.21.15.40:0/2338522419 shutdown_connections 2024-04-22T05:55:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3ee07d640 1 -- 172.21.15.40:0/2338522419 wait complete. 2024-04-22T05:55:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3ee07d640 1 Processor -- start 2024-04-22T05:55:24.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.566+0000 7fa3ee07d640 1 -- start start 2024-04-22T05:55:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3ee07d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e806e480 0x7fa3e80aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3ee07d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa3e80a7ab0 0x7fa3e813ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3ee07d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e813fe20 0x7fa3e8138520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3ee07d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e813b6e0 con 0x7fa3e813fe20 2024-04-22T05:55:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3ee07d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80aac30 con 0x7fa3e806e480 2024-04-22T05:55:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3ee07d640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e8139e20 con 0x7fa3e80a7ab0 2024-04-22T05:55:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3dffff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa3e80a7ab0 0x7fa3e813ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e813fe20 0x7fa3e8138520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3dffff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa3e80a7ab0 0x7fa3e813ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57734/0 (socket says 172.21.15.40:57734) 2024-04-22T05:55:24.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3dffff640 1 -- 172.21.15.40:0/3916926245 learned_addr learned my addr 172.21.15.40:0/3916926245 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:24.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e77fe640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e806e480 0x7fa3e80aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3dffff640 1 -- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e806e480 msgr2=0x7fa3e80aa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3dffff640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e806e480 0x7fa3e80aa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3dffff640 1 -- 172.21.15.40:0/3916926245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e813fe20 msgr2=0x7fa3e8138520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3dffff640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e813fe20 0x7fa3e8138520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3dffff640 1 -- 172.21.15.40:0/3916926245 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d0009420 con 0x7fa3e80a7ab0 2024-04-22T05:55:24.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e7fff640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e813fe20 0x7fa3e8138520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:24.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e77fe640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e806e480 0x7fa3e80aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:55:24.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3dffff640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa3e80a7ab0 0x7fa3e813ac40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa3d001a0b0 tx=0x7fa3d0019220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e4ff9640 1 -- 172.21.15.40:0/3916926245 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00ab850 con 0x7fa3e80a7ab0 2024-04-22T05:55:24.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e4ff9640 1 -- 172.21.15.40:0/3916926245 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d00a6040 con 0x7fa3e80a7ab0 2024-04-22T05:55:24.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3ee07d640 1 -- 172.21.15.40:0/3916926245 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e8134d00 con 0x7fa3e80a7ab0 2024-04-22T05:55:24.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e4ff9640 1 -- 172.21.15.40:0/3916926245 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00abc10 con 0x7fa3e80a7ab0 2024-04-22T05:55:24.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3ee07d640 1 -- 172.21.15.40:0/3916926245 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e81347b0 con 0x7fa3e80a7ab0 2024-04-22T05:55:24.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3ee07d640 1 -- 172.21.15.40:0/3916926245 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa3e806e890 con 0x7fa3e80a7ab0 2024-04-22T05:55:24.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e4ff9640 1 -- 172.21.15.40:0/3916926245 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3d00b4040 con 0x7fa3e80a7ab0 2024-04-22T05:55:24.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e4ff9640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa3c8066690 0x7fa3c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e4ff9640 1 -- 172.21.15.40:0/3916926245 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7fa3d011f0f0 con 0x7fa3e80a7ab0 2024-04-22T05:55:24.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e4ff9640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] conn(0x7fa3c806d7f0 0x7fa3c806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e4ff9640 1 -- 172.21.15.40:0/3916926245 --> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3c80702c0 con 0x7fa3c806d7f0 2024-04-22T05:55:24.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e4ff9640 1 -- 172.21.15.40:0/3916926245 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3d011f460 con 0x7fa3e80a7ab0 2024-04-22T05:55:24.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e77fe640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa3c8066690 0x7fa3c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.570+0000 7fa3e7fff640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] conn(0x7fa3c806d7f0 0x7fa3c806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.574+0000 7fa3e7fff640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] conn(0x7fa3c806d7f0 0x7fa3c806fc10 crc :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.574+0000 7fa3e77fe640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa3c8066690 0x7fa3c8068b50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009840 tx=0x7fa3d8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.574+0000 7fa3e4ff9640 1 -- 172.21.15.40:0/3916926245 <== osd.7 v2:172.21.15.62:6808/25138 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa3c80702c0 con 0x7fa3c806d7f0 2024-04-22T05:55:24.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.586+0000 7fa3ee07d640 1 -- 172.21.15.40:0/3916926245 --> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa3e80a9970 con 0x7fa3c806d7f0 2024-04-22T05:55:24.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.586+0000 7fa3e4ff9640 1 -- 172.21.15.40:0/3916926245 <== osd.7 v2:172.21.15.62:6808/25138 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa3e80a9970 con 0x7fa3c806d7f0 2024-04-22T05:55:24.589 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-22T05:55:24.589 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T05:55:24.589 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-22T05:55:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.586+0000 7fa3ee07d640 1 -- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] conn(0x7fa3c806d7f0 msgr2=0x7fa3c806fc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.586+0000 7fa3ee07d640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:6808/25138,v1:172.21.15.62:6809/25138] conn(0x7fa3c806d7f0 0x7fa3c806fc10 crc :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.586+0000 7fa3ee07d640 1 -- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa3c8066690 msgr2=0x7fa3c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.586+0000 7fa3ee07d640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa3c8066690 0x7fa3c8068b50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009840 tx=0x7fa3d8008040 comp rx=0 tx=0).stop 2024-04-22T05:55:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.586+0000 7fa3ee07d640 1 -- 172.21.15.40:0/3916926245 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa3e80a7ab0 msgr2=0x7fa3e813ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.586+0000 7fa3ee07d640 1 --2- 172.21.15.40:0/3916926245 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa3e80a7ab0 0x7fa3e813ac40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa3d001a0b0 tx=0x7fa3d0019220 comp rx=0 tx=0).stop 2024-04-22T05:55:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.586+0000 7fa3e7fff640 1 -- 172.21.15.40:0/3916926245 reap_dead start 2024-04-22T05:55:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.586+0000 7fa3ee07d640 1 -- 172.21.15.40:0/3916926245 shutdown_connections 2024-04-22T05:55:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.586+0000 7fa3ee07d640 1 -- 172.21.15.40:0/3916926245 >> 172.21.15.40:0/3916926245 conn(0x7fa3e80736e0 msgr2=0x7fa3e8137a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.586+0000 7fa3ee07d640 1 -- 172.21.15.40:0/3916926245 shutdown_connections 2024-04-22T05:55:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.586+0000 7fa3ee07d640 1 -- 172.21.15.40:0/3916926245 wait complete. 2024-04-22T05:55:24.604 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-22T05:55:24.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a07b4c640 1 Processor -- start 2024-04-22T05:55:24.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a07b4c640 1 -- start start 2024-04-22T05:55:24.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a07b4c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a00144bb0 0x7f0a0006dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a07b4c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0a00140e70 0x7f0a001453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a07b4c640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a000711b0 con 0x7f0a00144f90 2024-04-22T05:55:24.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a07b4c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a00071940 con 0x7f0a00144bb0 2024-04-22T05:55:24.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a07b4c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a00070ba0 con 0x7f0a00140e70 2024-04-22T05:55:24.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a060e9640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0a00140e70 0x7f0a001453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a058e8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a00144bb0 0x7f0a0006dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a060e9640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0a00140e70 0x7f0a001453d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:57748/0 (socket says 172.21.15.40:57748) 2024-04-22T05:55:24.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a060e9640 1 -- 172.21.15.40:0/1770965877 learned_addr learned my addr 172.21.15.40:0/1770965877 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:24.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a060e9640 1 -- 172.21.15.40:0/1770965877 >> v2:172.21.15.40:3300/0 conn(0x7f0a00144bb0 msgr2=0x7f0a0006dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:24.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a060e9640 1 --2- 172.21.15.40:0/1770965877 >> v2:172.21.15.40:3300/0 conn(0x7f0a00144bb0 0x7f0a0006dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a060e9640 1 -- 172.21.15.40:0/1770965877 >> v1:172.21.15.62:6789/0 conn(0x7f0a00144f90 legacy=0x7f0a00140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:24.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.718+0000 7f0a060e9640 1 -- 172.21.15.40:0/1770965877 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a00070d10 con 0x7f0a00140e70 2024-04-22T05:55:24.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a060e9640 1 --2- 172.21.15.40:0/1770965877 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0a00140e70 0x7f0a001453d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09f8009e50 tx=0x7f09f801c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37664eec5df691fd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a048e6640 1 -- 172.21.15.40:0/1770965877 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f80a5640 con 0x7f0a00140e70 2024-04-22T05:55:24.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a048e6640 1 -- 172.21.15.40:0/1770965877 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f80a57a0 con 0x7f0a00140e70 2024-04-22T05:55:24.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a048e6640 1 -- 172.21.15.40:0/1770965877 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f80a5b30 con 0x7f0a00140e70 2024-04-22T05:55:24.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 -- 172.21.15.40:0/1770965877 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0a00140e70 msgr2=0x7f0a001453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 --2- 172.21.15.40:0/1770965877 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0a00140e70 0x7f0a001453d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09f8009e50 tx=0x7f09f801c8b0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 -- 172.21.15.40:0/1770965877 shutdown_connections 2024-04-22T05:55:24.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 --2- 172.21.15.40:0/1770965877 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0a00140e70 0x7f0a001453d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 --2- 172.21.15.40:0/1770965877 >> v2:172.21.15.40:3300/0 conn(0x7f0a00144bb0 0x7f0a0006dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 -- 172.21.15.40:0/1770965877 >> 172.21.15.40:0/1770965877 conn(0x7f0a000b7440 msgr2=0x7f0a000b7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:24.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 -- 172.21.15.40:0/1770965877 shutdown_connections 2024-04-22T05:55:24.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 -- 172.21.15.40:0/1770965877 wait complete. 2024-04-22T05:55:24.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 Processor -- start 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 -- start start 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a00140e70 0x7f0a00135f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a00144bb0 0x7f0a001395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0a00144f90 0x7f0a001369d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a001365b0 con 0x7f0a00140e70 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a00137f30 con 0x7f0a00144bb0 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a07b4c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a00138210 con 0x7f0a00144f90 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a058e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a00140e70 0x7f0a00135f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a060e9640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0a00144f90 0x7f0a001369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a058e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a00140e70 0x7f0a00135f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34266/0 (socket says 172.21.15.40:34266) 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a058e8640 1 -- 172.21.15.40:0/819840865 learned_addr learned my addr 172.21.15.40:0/819840865 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a050e7640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a00144bb0 0x7f0a001395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a058e8640 1 -- 172.21.15.40:0/819840865 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0a00144f90 msgr2=0x7f0a001369d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a058e8640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0a00144f90 0x7f0a001369d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a058e8640 1 -- 172.21.15.40:0/819840865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a00144bb0 msgr2=0x7f0a001395f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a058e8640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a00144bb0 0x7f0a001395f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a058e8640 1 -- 172.21.15.40:0/819840865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f8009b30 con 0x7f0a00140e70 2024-04-22T05:55:24.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a060e9640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0a00144f90 0x7f0a001369d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:55:24.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f0a058e8640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a00140e70 0x7f0a00135f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f09f000e930 tx=0x7f09f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.722+0000 7f09f67fc640 1 -- 172.21.15.40:0/819840865 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0009690 con 0x7f0a00140e70 2024-04-22T05:55:24.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.726+0000 7f09f67fc640 1 -- 172.21.15.40:0/819840865 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f000ca00 con 0x7f0a00140e70 2024-04-22T05:55:24.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.726+0000 7f0a07b4c640 1 -- 172.21.15.40:0/819840865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a00136f10 con 0x7f0a00140e70 2024-04-22T05:55:24.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.726+0000 7f09f67fc640 1 -- 172.21.15.40:0/819840865 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f000cd70 con 0x7f0a00140e70 2024-04-22T05:55:24.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.726+0000 7f0a07b4c640 1 -- 172.21.15.40:0/819840865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a001301e0 con 0x7f0a00140e70 2024-04-22T05:55:24.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.726+0000 7f0a07b4c640 1 -- 172.21.15.40:0/819840865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a00067380 con 0x7f0a00140e70 2024-04-22T05:55:24.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.726+0000 7f09f67fc640 1 -- 172.21.15.40:0/819840865 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f09f0013470 con 0x7f0a00140e70 2024-04-22T05:55:24.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.726+0000 7f09f67fc640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f09e4066690 0x7f09e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:24.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.726+0000 7f09f67fc640 1 -- 172.21.15.40:0/819840865 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f09f0084b20 con 0x7f0a00140e70 2024-04-22T05:55:24.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.726+0000 7f0a050e7640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f09e4066690 0x7f09e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:24.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.726+0000 7f0a050e7640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f09e4066690 0x7f09e4068b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0a00138f60 tx=0x7f09fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:24.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.730+0000 7f09f67fc640 1 -- 172.21.15.40:0/819840865 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f09f00520b0 con 0x7f0a00140e70 2024-04-22T05:55:24.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.886+0000 7f0a07b4c640 1 -- 172.21.15.40:0/819840865 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a0004e690 con 0x7f09e4066690 2024-04-22T05:55:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.890+0000 7f09f67fc640 1 -- 172.21.15.40:0/819840865 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0a0004e690 con 0x7f09e4066690 2024-04-22T05:55:24.891 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 7 to deep-scrub 2024-04-22T05:55:24.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.890+0000 7f0a07b4c640 1 -- 172.21.15.40:0/819840865 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f09e4066690 msgr2=0x7f09e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.890+0000 7f0a07b4c640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f09e4066690 0x7f09e4068b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0a00138f60 tx=0x7f09fc008040 comp rx=0 tx=0).stop 2024-04-22T05:55:24.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.890+0000 7f0a07b4c640 1 -- 172.21.15.40:0/819840865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a00140e70 msgr2=0x7f0a00135f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:24.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.890+0000 7f0a07b4c640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a00140e70 0x7f0a00135f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f09f000e930 tx=0x7f09f000edf0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.890+0000 7f0a07b4c640 1 -- 172.21.15.40:0/819840865 shutdown_connections 2024-04-22T05:55:24.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.890+0000 7f0a07b4c640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f09e4066690 0x7f09e4068b50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.890+0000 7f0a07b4c640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a00144bb0 0x7f0a001395f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.890+0000 7f0a07b4c640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f0a00144f90 0x7f0a001369d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.890+0000 7f0a07b4c640 1 --2- 172.21.15.40:0/819840865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a00140e70 0x7f0a00135f80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:24.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.890+0000 7f0a07b4c640 1 -- 172.21.15.40:0/819840865 >> 172.21.15.40:0/819840865 conn(0x7f0a000b7440 msgr2=0x7f0a0006e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:24.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.890+0000 7f0a07b4c640 1 -- 172.21.15.40:0/819840865 shutdown_connections 2024-04-22T05:55:24.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:24.890+0000 7f0a07b4c640 1 -- 172.21.15.40:0/819840865 wait complete. 2024-04-22T05:55:24.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T05:55:25.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.030+0000 7fa19e4d3640 1 Processor -- start 2024-04-22T05:55:25.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.030+0000 7fa19e4d3640 1 -- start start 2024-04-22T05:55:25.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.030+0000 7fa19e4d3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1980b42d0 0x7fa19813e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:25.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.030+0000 7fa19e4d3640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1980b46b0 con 0x7fa1980a8ab0 2024-04-22T05:55:25.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.030+0000 7fa19e4d3640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1980b4ea0 con 0x7fa1981432d0 2024-04-22T05:55:25.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.030+0000 7fa19e4d3640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1980a9e90 con 0x7fa1980b42d0 2024-04-22T05:55:25.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.030+0000 7fa197fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1980b42d0 0x7fa19813e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:25.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.030+0000 7fa1977fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa1980a8ab0 0x7fa198142bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36856/0 (socket says 172.21.15.40:36856) 2024-04-22T05:55:25.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.030+0000 7fa1977fe640 1 -- 172.21.15.40:0/1407828605 learned_addr learned my addr 172.21.15.40:0/1407828605 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:55:25.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.030+0000 7fa197fff640 1 -- 172.21.15.40:0/1407828605 >> v1:172.21.15.62:6789/0 conn(0x7fa1981432d0 legacy=0x7fa198149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:25.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.030+0000 7fa197fff640 1 -- 172.21.15.40:0/1407828605 >> v1:172.21.15.40:6789/0 conn(0x7fa1980a8ab0 legacy=0x7fa198142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:25.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.030+0000 7fa196ffd640 1 -- 172.21.15.40:0/1407828605 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119949594 0 0) 0x7fa1980b46b0 con 0x7fa1980a8ab0 2024-04-22T05:55:25.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.030+0000 7fa197fff640 1 -- 172.21.15.40:0/1407828605 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198143700 con 0x7fa1980b42d0 2024-04-22T05:55:25.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa197fff640 1 --2- 172.21.15.40:0/1407828605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1980b42d0 0x7fa19813e620 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa1880097c0 tx=0x7fa18801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe0d103d9f44510 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:25.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa196ffd640 1 -- 172.21.15.40:0/1407828605 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1880ab600 con 0x7fa1980b42d0 2024-04-22T05:55:25.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa196ffd640 1 -- 172.21.15.40:0/1407828605 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1880a7050 con 0x7fa1980b42d0 2024-04-22T05:55:25.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa196ffd640 1 -- 172.21.15.40:0/1407828605 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1880ab980 con 0x7fa1980b42d0 2024-04-22T05:55:25.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 -- 172.21.15.40:0/1407828605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1980b42d0 msgr2=0x7fa19813e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:25.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 --2- 172.21.15.40:0/1407828605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1980b42d0 0x7fa19813e620 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa1880097c0 tx=0x7fa18801c390 comp rx=0 tx=0).stop 2024-04-22T05:55:25.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 -- 172.21.15.40:0/1407828605 shutdown_connections 2024-04-22T05:55:25.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 --2- 172.21.15.40:0/1407828605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1980b42d0 0x7fa19813e620 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:25.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 -- 172.21.15.40:0/1407828605 >> 172.21.15.40:0/1407828605 conn(0x7fa1980b1c60 msgr2=0x7fa1980b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:25.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 -- 172.21.15.40:0/1407828605 shutdown_connections 2024-04-22T05:55:25.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 -- 172.21.15.40:0/1407828605 wait complete. 2024-04-22T05:55:25.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 Processor -- start 2024-04-22T05:55:25.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 -- start start 2024-04-22T05:55:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1980a8ab0 0x7fa1981341c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa1980b42d0 0x7fa198141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1981432d0 0x7fa198141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198141d90 con 0x7fa1981432d0 2024-04-22T05:55:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198133760 con 0x7fa1980a8ab0 2024-04-22T05:55:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19813deb0 con 0x7fa1980b42d0 2024-04-22T05:55:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19ca70640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1981432d0 0x7fa198141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa1977fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa1980b42d0 0x7fa198141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19ca70640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1981432d0 0x7fa198141850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34286/0 (socket says 172.21.15.40:34286) 2024-04-22T05:55:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19ca70640 1 -- 172.21.15.40:0/2087014423 learned_addr learned my addr 172.21.15.40:0/2087014423 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa197fff640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1980a8ab0 0x7fa1981341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19ca70640 1 -- 172.21.15.40:0/2087014423 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa1980b42d0 msgr2=0x7fa198141310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19ca70640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa1980b42d0 0x7fa198141310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19ca70640 1 -- 172.21.15.40:0/2087014423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1980a8ab0 msgr2=0x7fa1981341c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19ca70640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1980a8ab0 0x7fa1981341c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19ca70640 1 -- 172.21.15.40:0/2087014423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa188009420 con 0x7fa1981432d0 2024-04-22T05:55:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa197fff640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1980a8ab0 0x7fa1981341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:55:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19ca70640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1981432d0 0x7fa198141850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa19000c730 tx=0x7fa19000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa194ff9640 1 -- 172.21.15.40:0/2087014423 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa190003b80 con 0x7fa1981432d0 2024-04-22T05:55:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa194ff9640 1 -- 172.21.15.40:0/2087014423 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa190003ce0 con 0x7fa1981432d0 2024-04-22T05:55:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 -- 172.21.15.40:0/2087014423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa198131840 con 0x7fa1981432d0 2024-04-22T05:55:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa194ff9640 1 -- 172.21.15.40:0/2087014423 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19000aa30 con 0x7fa1981432d0 2024-04-22T05:55:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.034+0000 7fa19e4d3640 1 -- 172.21.15.40:0/2087014423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1981d9720 con 0x7fa1981432d0 2024-04-22T05:55:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.038+0000 7fa19e4d3640 1 -- 172.21.15.40:0/2087014423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa198141030 con 0x7fa1981432d0 2024-04-22T05:55:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.038+0000 7fa194ff9640 1 -- 172.21.15.40:0/2087014423 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa19000ab90 con 0x7fa1981432d0 2024-04-22T05:55:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.038+0000 7fa194ff9640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa184066690 0x7fa184068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.038+0000 7fa194ff9640 1 -- 172.21.15.40:0/2087014423 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7fa190083de0 con 0x7fa1981432d0 2024-04-22T05:55:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.038+0000 7fa197fff640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa184066690 0x7fa184068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.038+0000 7fa197fff640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa184066690 0x7fa184068b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1880ad000 tx=0x7fa188008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:25.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.042+0000 7fa194ff9640 1 -- 172.21.15.40:0/2087014423 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7fa198141030 con 0x7fa1981432d0 2024-04-22T05:55:25.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.198+0000 7fa19e4d3640 1 -- 172.21.15.40:0/2087014423 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa19814db20 con 0x7fa184066690 2024-04-22T05:55:25.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.214+0000 7fa194ff9640 1 -- 172.21.15.40:0/2087014423 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+152657 (secure 0 0 0) 0x7fa1880ab760 con 0x7fa184066690 2024-04-22T05:55:25.215 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:55:25.215 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-22T05:55:25.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.214+0000 7fa19e4d3640 1 -- 172.21.15.40:0/2087014423 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa184066690 msgr2=0x7fa184068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:25.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.214+0000 7fa19e4d3640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa184066690 0x7fa184068b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1880ad000 tx=0x7fa188008040 comp rx=0 tx=0).stop 2024-04-22T05:55:25.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.214+0000 7fa19e4d3640 1 -- 172.21.15.40:0/2087014423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1981432d0 msgr2=0x7fa198141850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:25.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.214+0000 7fa19e4d3640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1981432d0 0x7fa198141850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa19000c730 tx=0x7fa19000cbf0 comp rx=0 tx=0).stop 2024-04-22T05:55:25.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.214+0000 7fa19e4d3640 1 -- 172.21.15.40:0/2087014423 shutdown_connections 2024-04-22T05:55:25.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.214+0000 7fa19e4d3640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fa184066690 0x7fa184068b50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:25.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.214+0000 7fa19e4d3640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1981432d0 0x7fa198141850 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:25.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.214+0000 7fa19e4d3640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fa1980b42d0 0x7fa198141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:25.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.214+0000 7fa19e4d3640 1 --2- 172.21.15.40:0/2087014423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1980a8ab0 0x7fa1981341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:25.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.214+0000 7fa19e4d3640 1 -- 172.21.15.40:0/2087014423 >> 172.21.15.40:0/2087014423 conn(0x7fa1980b1c60 msgr2=0x7fa198139e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:25.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.214+0000 7fa19e4d3640 1 -- 172.21.15.40:0/2087014423 shutdown_connections 2024-04-22T05:55:25.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:25.214+0000 7fa19e4d3640 1 -- 172.21.15.40:0/2087014423 wait complete. 2024-04-22T05:55:25.231 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-04-22T05:55:25.026801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3568096024,"num_objects":879,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":879,"num_whiteouts":0,"num_read":2676,"num_read_kb":2676,"num_write":157303,"num_write_kb":5628657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158677,"ondisk_log_size":158677,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9131776,"kb_used_data":6638756,"kb_used_omap":12,"kb_used_meta":2492915,"kb_avail":745842944,"statfs":{"total":773094113280,"available":763743174656,"internally_reserved":0,"allocated":6798086144,"data_stored":6835410060,"data_compressed":194286840,"data_compressed_allocated":202113024,"data_compressed_original":244916224,"omap_allocated":12744,"internal_metadata":2552745528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8720,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":66,"num_read_kb":71,"num_write":17,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003267"},"pg_stats":[{"pgid":"3.1f","version":"21'3322","reported_seq":3350,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.181289+0000","last_change":"2024-04-22T05:50:47.121259+0000","last_active":"2024-04-22T05:55:18.181289+0000","last_peered":"2024-04-22T05:55:18.181289+0000","last_clean":"2024-04-22T05:55:18.181289+0000","last_became_active":"2024-04-22T05:50:47.120501+0000","last_became_peered":"2024-04-22T05:50:47.120501+0000","last_unstale":"2024-04-22T05:55:18.181289+0000","last_undegraded":"2024-04-22T05:55:18.181289+0000","last_fullsized":"2024-04-22T05:55:18.181289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3322,"log_dups_size":0,"ondisk_log_size":3322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109323296,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":3347,"num_write_kb":168857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'5497","reported_seq":5520,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.200857+0000","last_change":"2024-04-22T05:50:47.118054+0000","last_active":"2024-04-22T05:55:18.200857+0000","last_peered":"2024-04-22T05:55:18.200857+0000","last_clean":"2024-04-22T05:55:18.200857+0000","last_became_active":"2024-04-22T05:50:47.117274+0000","last_became_peered":"2024-04-22T05:50:47.117274+0000","last_unstale":"2024-04-22T05:55:18.200857+0000","last_undegraded":"2024-04-22T05:55:18.200857+0000","last_fullsized":"2024-04-22T05:55:18.200857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5497,"log_dups_size":0,"ondisk_log_size":5497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125960704,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":28,"num_read_kb":84,"num_write":5488,"num_write_kb":185156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"23'3418","reported_seq":3443,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.201673+0000","last_change":"2024-04-22T05:50:47.122061+0000","last_active":"2024-04-22T05:55:18.201673+0000","last_peered":"2024-04-22T05:55:18.201673+0000","last_clean":"2024-04-22T05:55:18.201673+0000","last_became_active":"2024-04-22T05:50:47.120601+0000","last_became_peered":"2024-04-22T05:50:47.120601+0000","last_unstale":"2024-04-22T05:55:18.201673+0000","last_undegraded":"2024-04-22T05:55:18.201673+0000","last_fullsized":"2024-04-22T05:55:18.201673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3418,"log_dups_size":0,"ondisk_log_size":3418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72812200,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":3406,"num_write_kb":94339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2981","reported_seq":2995,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.185853+0000","last_change":"2024-04-22T05:50:47.117092+0000","last_active":"2024-04-22T05:55:18.185853+0000","last_peered":"2024-04-22T05:55:18.185853+0000","last_clean":"2024-04-22T05:55:18.185853+0000","last_became_active":"2024-04-22T05:50:47.115803+0000","last_became_peered":"2024-04-22T05:50:47.115803+0000","last_unstale":"2024-04-22T05:55:18.185853+0000","last_undegraded":"2024-04-22T05:55:18.185853+0000","last_fullsized":"2024-04-22T05:55:18.185853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":2981,"log_dups_size":0,"ondisk_log_size":2981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89522176,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2976,"num_write_kb":149441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"23'5222","reported_seq":5264,"reported_epoch":23,"state":"active+clean+scrubbing+deep","last_fresh":"2024-04-22T05:55:24.055897+0000","last_change":"2024-04-22T05:55:24.054771+0000","last_active":"2024-04-22T05:55:24.055897+0000","last_peered":"2024-04-22T05:55:24.055897+0000","last_clean":"2024-04-22T05:55:24.055897+0000","last_became_active":"2024-04-22T05:50:47.123073+0000","last_became_peered":"2024-04-22T05:50:47.123073+0000","last_unstale":"2024-04-22T05:55:24.055897+0000","last_undegraded":"2024-04-22T05:55:24.055897+0000","last_fullsized":"2024-04-22T05:55:24.055897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5222,"log_dups_size":0,"ondisk_log_size":5222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150994944,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":90,"num_read_kb":78,"num_write":5209,"num_write_kb":201924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"23'6503","reported_seq":6525,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.192148+0000","last_change":"2024-04-22T05:50:47.118816+0000","last_active":"2024-04-22T05:55:18.192148+0000","last_peered":"2024-04-22T05:55:18.192148+0000","last_clean":"2024-04-22T05:55:18.192148+0000","last_became_active":"2024-04-22T05:50:47.118038+0000","last_became_peered":"2024-04-22T05:50:47.118038+0000","last_unstale":"2024-04-22T05:55:18.192148+0000","last_undegraded":"2024-04-22T05:55:18.192148+0000","last_fullsized":"2024-04-22T05:55:18.192148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6503,"log_dups_size":0,"ondisk_log_size":6503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113311744,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":6483,"num_write_kb":139343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"23'4755","reported_seq":4769,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.195328+0000","last_change":"2024-04-22T05:50:47.114124+0000","last_active":"2024-04-22T05:55:18.195328+0000","last_peered":"2024-04-22T05:55:18.195328+0000","last_clean":"2024-04-22T05:55:18.195328+0000","last_became_active":"2024-04-22T05:50:47.113368+0000","last_became_peered":"2024-04-22T05:50:47.113368+0000","last_unstale":"2024-04-22T05:55:18.195328+0000","last_undegraded":"2024-04-22T05:55:18.195328+0000","last_fullsized":"2024-04-22T05:55:18.195328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4755,"log_dups_size":0,"ondisk_log_size":4755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112566272,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":123245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"22'8053","reported_seq":8074,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.175584+0000","last_change":"2024-04-22T05:50:47.114390+0000","last_active":"2024-04-22T05:55:18.175584+0000","last_peered":"2024-04-22T05:55:18.175584+0000","last_clean":"2024-04-22T05:55:18.175584+0000","last_became_active":"2024-04-22T05:50:47.113569+0000","last_became_peered":"2024-04-22T05:50:47.113569+0000","last_unstale":"2024-04-22T05:55:18.175584+0000","last_undegraded":"2024-04-22T05:55:18.175584+0000","last_fullsized":"2024-04-22T05:55:18.175584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":8053,"log_dups_size":0,"ondisk_log_size":8053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":8065,"num_write_kb":185808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"22'6472","reported_seq":6497,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.182129+0000","last_change":"2024-04-22T05:50:47.122110+0000","last_active":"2024-04-22T05:55:18.182129+0000","last_peered":"2024-04-22T05:55:18.182129+0000","last_clean":"2024-04-22T05:55:18.182129+0000","last_became_active":"2024-04-22T05:50:47.121332+0000","last_became_peered":"2024-04-22T05:50:47.121332+0000","last_unstale":"2024-04-22T05:55:18.182129+0000","last_undegraded":"2024-04-22T05:55:18.182129+0000","last_fullsized":"2024-04-22T05:55:18.182129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6472,"log_dups_size":0,"ondisk_log_size":6472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111009792,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":6495,"num_write_kb":211703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"23'3762","reported_seq":3778,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.200671+0000","last_change":"2024-04-22T05:50:47.120024+0000","last_active":"2024-04-22T05:55:18.200671+0000","last_peered":"2024-04-22T05:55:18.200671+0000","last_clean":"2024-04-22T05:55:18.200671+0000","last_became_active":"2024-04-22T05:50:47.119031+0000","last_became_peered":"2024-04-22T05:50:47.119031+0000","last_unstale":"2024-04-22T05:55:18.200671+0000","last_undegraded":"2024-04-22T05:55:18.200671+0000","last_fullsized":"2024-04-22T05:55:18.200671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3762,"log_dups_size":0,"ondisk_log_size":3762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":3747,"num_write_kb":173061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"22'7962","reported_seq":7975,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:19.971677+0000","last_change":"2024-04-22T05:50:47.119142+0000","last_active":"2024-04-22T05:54:19.971677+0000","last_peered":"2024-04-22T05:54:19.971677+0000","last_clean":"2024-04-22T05:54:19.971677+0000","last_became_active":"2024-04-22T05:50:47.118399+0000","last_became_peered":"2024-04-22T05:50:47.118399+0000","last_unstale":"2024-04-22T05:54:19.971677+0000","last_undegraded":"2024-04-22T05:54:19.971677+0000","last_fullsized":"2024-04-22T05:54:19.971677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":7962,"log_dups_size":0,"ondisk_log_size":7962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:42:14.831153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7958,"num_write_kb":234882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"23'7048","reported_seq":7091,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.201699+0000","last_change":"2024-04-22T05:50:47.131544+0000","last_active":"2024-04-22T05:55:18.201699+0000","last_peered":"2024-04-22T05:55:18.201699+0000","last_clean":"2024-04-22T05:55:18.201699+0000","last_became_active":"2024-04-22T05:50:47.130842+0000","last_became_peered":"2024-04-22T05:50:47.130842+0000","last_unstale":"2024-04-22T05:55:18.201699+0000","last_undegraded":"2024-04-22T05:55:18.201699+0000","last_fullsized":"2024-04-22T05:55:18.201699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":7048,"log_dups_size":0,"ondisk_log_size":7048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149815296,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":118,"num_read_kb":94,"num_write":7055,"num_write_kb":232332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:55:04.041975+0000","last_change":"2024-04-22T05:50:41.050585+0000","last_active":"2024-04-22T05:55:04.041975+0000","last_peered":"2024-04-22T05:55:04.041975+0000","last_clean":"2024-04-22T05:55:04.041975+0000","last_became_active":"2024-04-22T05:50:41.049696+0000","last_became_peered":"2024-04-22T05:50:41.049696+0000","last_unstale":"2024-04-22T05:55:04.041975+0000","last_undegraded":"2024-04-22T05:55:04.041975+0000","last_fullsized":"2024-04-22T05:55:04.041975+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:00:56.754144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'4444","reported_seq":4479,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.196814+0000","last_change":"2024-04-22T05:50:47.114576+0000","last_active":"2024-04-22T05:55:18.196814+0000","last_peered":"2024-04-22T05:55:18.196814+0000","last_clean":"2024-04-22T05:55:18.196814+0000","last_became_active":"2024-04-22T05:50:47.113838+0000","last_became_peered":"2024-04-22T05:50:47.113838+0000","last_unstale":"2024-04-22T05:55:18.196814+0000","last_undegraded":"2024-04-22T05:55:18.196814+0000","last_fullsized":"2024-04-22T05:55:18.196814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4444,"log_dups_size":0,"ondisk_log_size":4444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120194896,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":4449,"num_write_kb":155490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.100571+0000","last_change":"2024-04-22T05:50:41.048959+0000","last_active":"2024-04-22T05:55:18.100571+0000","last_peered":"2024-04-22T05:55:18.100571+0000","last_clean":"2024-04-22T05:55:18.100571+0000","last_became_active":"2024-04-22T05:50:41.048628+0000","last_became_peered":"2024-04-22T05:50:41.048628+0000","last_unstale":"2024-04-22T05:55:18.100571+0000","last_undegraded":"2024-04-22T05:55:18.100571+0000","last_fullsized":"2024-04-22T05:55:18.100571+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'3254","reported_seq":3271,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.199229+0000","last_change":"2024-04-22T05:50:47.130110+0000","last_active":"2024-04-22T05:55:18.199229+0000","last_peered":"2024-04-22T05:55:18.199229+0000","last_clean":"2024-04-22T05:55:18.199229+0000","last_became_active":"2024-04-22T05:50:47.129322+0000","last_became_peered":"2024-04-22T05:50:47.129322+0000","last_unstale":"2024-04-22T05:55:18.199229+0000","last_undegraded":"2024-04-22T05:55:18.199229+0000","last_fullsized":"2024-04-22T05:55:18.199229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3254,"log_dups_size":0,"ondisk_log_size":3254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":3214,"num_write_kb":155139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:53.173287+0000","last_change":"2024-04-22T05:50:41.341624+0000","last_active":"2024-04-22T05:54:53.173287+0000","last_peered":"2024-04-22T05:54:53.173287+0000","last_clean":"2024-04-22T05:54:53.173287+0000","last_became_active":"2024-04-22T05:50:41.340643+0000","last_became_peered":"2024-04-22T05:50:41.340643+0000","last_unstale":"2024-04-22T05:54:53.173287+0000","last_undegraded":"2024-04-22T05:54:53.173287+0000","last_fullsized":"2024-04-22T05:54:53.173287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:47:12.897727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'6361","reported_seq":6382,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.100601+0000","last_change":"2024-04-22T05:50:47.172954+0000","last_active":"2024-04-22T05:55:18.100601+0000","last_peered":"2024-04-22T05:55:18.100601+0000","last_clean":"2024-04-22T05:55:18.100601+0000","last_became_active":"2024-04-22T05:50:47.172121+0000","last_became_peered":"2024-04-22T05:50:47.172121+0000","last_unstale":"2024-04-22T05:55:18.100601+0000","last_undegraded":"2024-04-22T05:55:18.100601+0000","last_fullsized":"2024-04-22T05:55:18.100601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6361,"log_dups_size":0,"ondisk_log_size":6361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":26,"num_read_kb":18,"num_write":6371,"num_write_kb":148611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.101012+0000","last_change":"2024-04-22T05:50:41.341548+0000","last_active":"2024-04-22T05:55:18.101012+0000","last_peered":"2024-04-22T05:55:18.101012+0000","last_clean":"2024-04-22T05:55:18.101012+0000","last_became_active":"2024-04-22T05:50:41.341254+0000","last_became_peered":"2024-04-22T05:50:41.341254+0000","last_unstale":"2024-04-22T05:55:18.101012+0000","last_undegraded":"2024-04-22T05:55:18.101012+0000","last_fullsized":"2024-04-22T05:55:18.101012+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"21'3305","reported_seq":3331,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.188078+0000","last_change":"2024-04-22T05:50:47.121899+0000","last_active":"2024-04-22T05:55:18.188078+0000","last_peered":"2024-04-22T05:55:18.188078+0000","last_clean":"2024-04-22T05:55:18.188078+0000","last_became_active":"2024-04-22T05:50:47.120072+0000","last_became_peered":"2024-04-22T05:50:47.120072+0000","last_unstale":"2024-04-22T05:55:18.188078+0000","last_undegraded":"2024-04-22T05:55:18.188078+0000","last_fullsized":"2024-04-22T05:55:18.188078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3305,"log_dups_size":0,"ondisk_log_size":3305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99223376,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":3325,"num_write_kb":171289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'3629","reported_seq":3653,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.111433+0000","last_change":"2024-04-22T05:50:47.125635+0000","last_active":"2024-04-22T05:55:18.111433+0000","last_peered":"2024-04-22T05:55:18.111433+0000","last_clean":"2024-04-22T05:55:18.111433+0000","last_became_active":"2024-04-22T05:50:47.124841+0000","last_became_peered":"2024-04-22T05:50:47.124841+0000","last_unstale":"2024-04-22T05:55:18.111433+0000","last_undegraded":"2024-04-22T05:55:18.111433+0000","last_fullsized":"2024-04-22T05:55:18.111433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3629,"log_dups_size":0,"ondisk_log_size":3629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":3649,"num_write_kb":178556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.185052+0000","last_change":"2024-04-22T05:50:41.051239+0000","last_active":"2024-04-22T05:55:18.185052+0000","last_peered":"2024-04-22T05:55:18.185052+0000","last_clean":"2024-04-22T05:55:18.185052+0000","last_became_active":"2024-04-22T05:50:41.050711+0000","last_became_peered":"2024-04-22T05:50:41.050711+0000","last_unstale":"2024-04-22T05:55:18.185052+0000","last_undegraded":"2024-04-22T05:55:18.185052+0000","last_fullsized":"2024-04-22T05:55:18.185052+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"23'5390","reported_seq":5408,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.193143+0000","last_change":"2024-04-22T05:50:47.173005+0000","last_active":"2024-04-22T05:55:18.193143+0000","last_peered":"2024-04-22T05:55:18.193143+0000","last_clean":"2024-04-22T05:55:18.193143+0000","last_became_active":"2024-04-22T05:50:47.172218+0000","last_became_peered":"2024-04-22T05:50:47.172218+0000","last_unstale":"2024-04-22T05:55:18.193143+0000","last_undegraded":"2024-04-22T05:55:18.193143+0000","last_fullsized":"2024-04-22T05:55:18.193143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5390,"log_dups_size":0,"ondisk_log_size":5390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138609688,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":5358,"num_write_kb":177059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:50.028717+0000","last_change":"2024-04-22T05:50:41.051710+0000","last_active":"2024-04-22T05:54:50.028717+0000","last_peered":"2024-04-22T05:54:50.028717+0000","last_clean":"2024-04-22T05:54:50.028717+0000","last_became_active":"2024-04-22T05:50:41.051259+0000","last_became_peered":"2024-04-22T05:50:41.051259+0000","last_unstale":"2024-04-22T05:54:50.028717+0000","last_undegraded":"2024-04-22T05:54:50.028717+0000","last_fullsized":"2024-04-22T05:54:50.028717+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:41:46.406807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"23'4950","reported_seq":5066,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.198142+0000","last_change":"2024-04-22T05:50:47.120166+0000","last_active":"2024-04-22T05:55:18.198142+0000","last_peered":"2024-04-22T05:55:18.198142+0000","last_clean":"2024-04-22T05:55:18.198142+0000","last_became_active":"2024-04-22T05:50:47.119414+0000","last_became_peered":"2024-04-22T05:50:47.119414+0000","last_unstale":"2024-04-22T05:55:18.198142+0000","last_undegraded":"2024-04-22T05:55:18.198142+0000","last_fullsized":"2024-04-22T05:55:18.198142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4950,"log_dups_size":0,"ondisk_log_size":4950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124520800,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":4927,"num_write_kb":205619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:43.171572+0000","last_change":"2024-04-22T05:50:41.045192+0000","last_active":"2024-04-22T05:54:43.171572+0000","last_peered":"2024-04-22T05:54:43.171572+0000","last_clean":"2024-04-22T05:54:43.171572+0000","last_became_active":"2024-04-22T05:50:41.044865+0000","last_became_peered":"2024-04-22T05:50:41.044865+0000","last_unstale":"2024-04-22T05:54:43.171572+0000","last_undegraded":"2024-04-22T05:54:43.171572+0000","last_fullsized":"2024-04-22T05:54:43.171572+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:37:43.523824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"22'5431","reported_seq":5447,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.175724+0000","last_change":"2024-04-22T05:50:47.114255+0000","last_active":"2024-04-22T05:55:18.175724+0000","last_peered":"2024-04-22T05:55:18.175724+0000","last_clean":"2024-04-22T05:55:18.175724+0000","last_became_active":"2024-04-22T05:50:47.113405+0000","last_became_peered":"2024-04-22T05:50:47.113405+0000","last_unstale":"2024-04-22T05:55:18.175724+0000","last_undegraded":"2024-04-22T05:55:18.175724+0000","last_fullsized":"2024-04-22T05:55:18.175724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128647688,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5434,"num_write_kb":203340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.185296+0000","last_change":"2024-04-22T05:50:41.052651+0000","last_active":"2024-04-22T05:55:18.185296+0000","last_peered":"2024-04-22T05:55:18.185296+0000","last_clean":"2024-04-22T05:55:18.185296+0000","last_became_active":"2024-04-22T05:50:41.052349+0000","last_became_peered":"2024-04-22T05:50:41.052349+0000","last_unstale":"2024-04-22T05:55:18.185296+0000","last_undegraded":"2024-04-22T05:55:18.185296+0000","last_fullsized":"2024-04-22T05:55:18.185296+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:40.020947+0000","last_clean_scrub_stamp":"2024-04-22T05:50:40.020947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:43.171606+0000","last_change":"2024-04-22T05:50:40.047233+0000","last_active":"2024-04-22T05:54:43.171606+0000","last_peered":"2024-04-22T05:54:43.171606+0000","last_clean":"2024-04-22T05:54:43.171606+0000","last_became_active":"2024-04-22T05:50:40.046329+0000","last_became_peered":"2024-04-22T05:50:40.046329+0000","last_unstale":"2024-04-22T05:54:43.171606+0000","last_undegraded":"2024-04-22T05:54:43.171606+0000","last_fullsized":"2024-04-22T05:54:43.171606+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:39.011105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:39.011105+0000","last_clean_scrub_stamp":"2024-04-22T05:50:39.011105+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:16:02.587654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"23'3932","reported_seq":3950,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.202394+0000","last_change":"2024-04-22T05:50:47.116127+0000","last_active":"2024-04-22T05:55:18.202394+0000","last_peered":"2024-04-22T05:55:18.202394+0000","last_clean":"2024-04-22T05:55:18.202394+0000","last_became_active":"2024-04-22T05:50:47.115449+0000","last_became_peered":"2024-04-22T05:50:47.115449+0000","last_unstale":"2024-04-22T05:55:18.202394+0000","last_undegraded":"2024-04-22T05:55:18.202394+0000","last_fullsized":"2024-04-22T05:55:18.202394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3932,"log_dups_size":0,"ondisk_log_size":3932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":3867,"num_write_kb":173187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"23'10225","reported_seq":10355,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.247455+0000","last_change":"2024-04-22T05:50:47.173998+0000","last_active":"2024-04-22T05:55:18.247455+0000","last_peered":"2024-04-22T05:55:18.247455+0000","last_clean":"2024-04-22T05:55:18.247455+0000","last_became_active":"2024-04-22T05:50:47.173440+0000","last_became_peered":"2024-04-22T05:50:47.173440+0000","last_unstale":"2024-04-22T05:55:18.247455+0000","last_undegraded":"2024-04-22T05:55:18.247455+0000","last_fullsized":"2024-04-22T05:55:18.247455+0000","mapping_epoch":19,"log_start":"21'200","ondisk_log_start":"21'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":381,"num_read_kb":346,"num_write":8425,"num_write_kb":237093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"23'5036","reported_seq":5050,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.099893+0000","last_change":"2024-04-22T05:50:47.121542+0000","last_active":"2024-04-22T05:55:18.099893+0000","last_peered":"2024-04-22T05:55:18.099893+0000","last_clean":"2024-04-22T05:55:18.099893+0000","last_became_active":"2024-04-22T05:50:47.120541+0000","last_became_peered":"2024-04-22T05:50:47.120541+0000","last_unstale":"2024-04-22T05:55:18.099893+0000","last_undegraded":"2024-04-22T05:55:18.099893+0000","last_fullsized":"2024-04-22T05:55:18.099893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:07:12.244690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5009,"num_write_kb":193539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"21'4714","reported_seq":4732,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.181407+0000","last_change":"2024-04-22T05:50:47.123811+0000","last_active":"2024-04-22T05:55:18.181407+0000","last_peered":"2024-04-22T05:55:18.181407+0000","last_clean":"2024-04-22T05:55:18.181407+0000","last_became_active":"2024-04-22T05:50:47.123115+0000","last_became_peered":"2024-04-22T05:50:47.123115+0000","last_unstale":"2024-04-22T05:55:18.181407+0000","last_undegraded":"2024-04-22T05:55:18.181407+0000","last_fullsized":"2024-04-22T05:55:18.181407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600592,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":4720,"num_write_kb":218508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"23'4378","reported_seq":4397,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:19.091396+0000","last_change":"2024-04-22T05:50:47.116574+0000","last_active":"2024-04-22T05:55:19.091396+0000","last_peered":"2024-04-22T05:55:19.091396+0000","last_clean":"2024-04-22T05:55:19.091396+0000","last_became_active":"2024-04-22T05:50:47.114834+0000","last_became_peered":"2024-04-22T05:50:47.114834+0000","last_unstale":"2024-04-22T05:55:19.091396+0000","last_undegraded":"2024-04-22T05:55:19.091396+0000","last_fullsized":"2024-04-22T05:55:19.091396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4378,"log_dups_size":0,"ondisk_log_size":4378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":4338,"num_write_kb":218182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'3554","reported_seq":3568,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.183995+0000","last_change":"2024-04-22T05:50:47.117704+0000","last_active":"2024-04-22T05:55:18.183995+0000","last_peered":"2024-04-22T05:55:18.183995+0000","last_clean":"2024-04-22T05:55:18.183995+0000","last_became_active":"2024-04-22T05:50:47.113260+0000","last_became_peered":"2024-04-22T05:50:47.113260+0000","last_unstale":"2024-04-22T05:55:18.183995+0000","last_undegraded":"2024-04-22T05:55:18.183995+0000","last_fullsized":"2024-04-22T05:55:18.183995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:54:55.343145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104202240,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3525,"num_write_kb":160132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'3632","reported_seq":3738,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.211410+0000","last_change":"2024-04-22T05:50:47.123901+0000","last_active":"2024-04-22T05:55:18.211410+0000","last_peered":"2024-04-22T05:55:18.211410+0000","last_clean":"2024-04-22T05:55:18.211410+0000","last_became_active":"2024-04-22T05:50:47.122822+0000","last_became_peered":"2024-04-22T05:50:47.122822+0000","last_unstale":"2024-04-22T05:55:18.211410+0000","last_undegraded":"2024-04-22T05:55:18.211410+0000","last_fullsized":"2024-04-22T05:55:18.211410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3632,"log_dups_size":0,"ondisk_log_size":3632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:05.611537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":300,"num_read_kb":235,"num_write":3883,"num_write_kb":108782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'4331","reported_seq":4369,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.193344+0000","last_change":"2024-04-22T05:50:47.118515+0000","last_active":"2024-04-22T05:55:18.193344+0000","last_peered":"2024-04-22T05:55:18.193344+0000","last_clean":"2024-04-22T05:55:18.193344+0000","last_became_active":"2024-04-22T05:50:47.117734+0000","last_became_peered":"2024-04-22T05:50:47.117734+0000","last_unstale":"2024-04-22T05:55:18.193344+0000","last_undegraded":"2024-04-22T05:55:18.193344+0000","last_fullsized":"2024-04-22T05:55:18.193344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:33:43.659487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126091776,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":92,"num_read_kb":134,"num_write":4330,"num_write_kb":222405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"23'3197","reported_seq":3210,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.194522+0000","last_change":"2024-04-22T05:50:47.125698+0000","last_active":"2024-04-22T05:55:18.194522+0000","last_peered":"2024-04-22T05:55:18.194522+0000","last_clean":"2024-04-22T05:55:18.194522+0000","last_became_active":"2024-04-22T05:50:47.124711+0000","last_became_peered":"2024-04-22T05:50:47.124711+0000","last_unstale":"2024-04-22T05:55:18.194522+0000","last_undegraded":"2024-04-22T05:55:18.194522+0000","last_fullsized":"2024-04-22T05:55:18.194522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3197,"log_dups_size":0,"ondisk_log_size":3197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:06:16.139033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3149,"num_write_kb":120072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"21'3697","reported_seq":4016,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-22T05:54:02.080934+0000","last_change":"2024-04-22T05:50:47.116263+0000","last_active":"2024-04-22T05:54:02.080934+0000","last_peered":"2024-04-22T05:54:02.080934+0000","last_clean":"2024-04-22T05:54:02.080934+0000","last_became_active":"2024-04-22T05:50:47.115535+0000","last_became_peered":"2024-04-22T05:50:47.115535+0000","last_unstale":"2024-04-22T05:54:02.080934+0000","last_undegraded":"2024-04-22T05:54:02.080934+0000","last_fullsized":"2024-04-22T05:54:02.080934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:20.378220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92406272,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1026,"num_read_kb":919,"num_write":3912,"num_write_kb":173447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"23'3456","reported_seq":3484,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:23.312666+0000","last_change":"2024-04-22T05:55:23.312857+0000","last_active":"2024-04-22T05:55:23.312666+0000","last_peered":"2024-04-22T05:55:23.312666+0000","last_clean":"2024-04-22T05:55:23.312666+0000","last_became_active":"2024-04-22T05:50:47.129345+0000","last_became_peered":"2024-04-22T05:50:47.129345+0000","last_unstale":"2024-04-22T05:55:23.312666+0000","last_undegraded":"2024-04-22T05:55:23.312666+0000","last_fullsized":"2024-04-22T05:55:23.312666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3456","last_scrub_stamp":"2024-04-22T05:55:23.312622+0000","last_deep_scrub":"23'3456","last_deep_scrub_stamp":"2024-04-22T05:55:23.312622+0000","last_clean_scrub_stamp":"2024-04-22T05:55:23.312622+0000","objects_scrubbed":18,"log_size":3456,"log_dups_size":0,"ondisk_log_size":3456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:34:38.998900+0000","scrub_duration":0.21609183200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":71,"num_write":3499,"num_write_kb":148421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"23'6849","reported_seq":6863,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.099999+0000","last_change":"2024-04-22T05:50:47.127371+0000","last_active":"2024-04-22T05:55:18.099999+0000","last_peered":"2024-04-22T05:55:18.099999+0000","last_clean":"2024-04-22T05:55:18.099999+0000","last_became_active":"2024-04-22T05:50:47.126563+0000","last_became_peered":"2024-04-22T05:50:47.126563+0000","last_unstale":"2024-04-22T05:55:18.099999+0000","last_undegraded":"2024-04-22T05:55:18.099999+0000","last_fullsized":"2024-04-22T05:55:18.099999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6849,"log_dups_size":0,"ondisk_log_size":6849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:16:53.013583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1,"num_read_kb":64,"num_write":6824,"num_write_kb":157279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3567505656,"num_objects":877,"num_object_clones":0,"num_object_copies":1754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":877,"num_whiteouts":0,"num_read":2560,"num_read_kb":2576,"num_write":157144,"num_write_kb":5626241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6796083200,"data_stored":6833661048,"data_compressed":194275682,"data_compressed_allocated":202047488,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":158560,"ondisk_log_size":158560,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11158,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":14,"seq":60129542202,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591964,"kb_used_data":408272,"kb_used_omap":1,"kb_used_meta":183678,"kb_avail":93779876,"statfs":{"total":96636764160,"available":96030593024,"internally_reserved":0,"allocated":418070528,"data_stored":419212919,"data_compressed":12754442,"data_compressed_allocated":13279232,"data_compressed_original":14893056,"omap_allocated":1594,"internal_metadata":188086726},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.84599999999999997,"15min":0.78800000000000003},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.95799999999999996,"5min":20.890000000000001,"15min":20.890000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.622,"15min":0.66000000000000003},"min":{"1min":0.22600000000000001,"5min":0.185,"15min":0.185},"max":{"1min":1.032,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.89100000000000001}]},{"osd":1,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66400000000000003,"15min":0.67200000000000004},"min":{"1min":0.24199999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.221,"15min":0.221},"max":{"1min":1.04,"5min":1.218,"15min":1.218},"last":0.57099999999999995}]},{"osd":3,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.747,"15min":0.86499999999999999},"min":{"1min":0.26100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.92700000000000005,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.625},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.82099999999999995,"15min":0.88700000000000001},"min":{"1min":0.33000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":1.111,"5min":3.27,"15min":3.27},"last":0.76500000000000001}]},{"osd":4,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81599999999999995,"15min":0.80200000000000005},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.86299999999999999,"15min":0.85499999999999998},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.27,"5min":1.54,"15min":1.54},"last":0.86699999999999999}]},{"osd":5,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82999999999999996,"15min":0.84599999999999997},"min":{"1min":0.39100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.389,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.36699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.6439999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.1180000000000001}]},{"osd":6,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.92800000000000005,"15min":0.95399999999999996},"min":{"1min":0.41699999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.204,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94499999999999995,"15min":0.97399999999999998},"min":{"1min":0.39400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.597,"5min":1.651,"15min":1.651},"last":1.1919999999999999}]},{"osd":7,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.2869999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.091},{"interface":"front","average":{"1min":0.879,"5min":1.0920000000000001,"15min":1.02},"min":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4750000000000001,"5min":17.995999999999999,"15min":17.995999999999999},"last":1.0640000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418592,"kb_used_data":311188,"kb_used_omap":1,"kb_used_meta":107390,"kb_avail":93953248,"statfs":{"total":96636764160,"available":96208125952,"internally_reserved":0,"allocated":318656512,"data_stored":318798411,"data_compressed":4811634,"data_compressed_allocated":5001216,"data_compressed_original":5484544,"omap_allocated":1594,"internal_metadata":109967814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":1.506,"15min":1.8380000000000001},"min":{"1min":0.373,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.0720000000000001,"5min":34.100000000000001,"15min":34.100000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":1.0589999999999999,"15min":0.85199999999999998},"min":{"1min":0.439,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.131,"5min":30.306000000000001,"15min":30.306000000000001},"last":0.70999999999999996}]},{"osd":1,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.79200000000000004,"15min":0.73799999999999999},"min":{"1min":0.47799999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.2849999999999999,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.81599999999999995,"15min":0.77600000000000002},"min":{"1min":0.32600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":1.2430000000000001,"5min":1.74,"15min":1.74},"last":0.76500000000000001}]},{"osd":2,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.88500000000000001,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.216,"15min":0.216},"max":{"1min":1.3009999999999999,"5min":2.488,"15min":2.488},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.92700000000000005,"15min":0.88700000000000001},"min":{"1min":0.503,"5min":0.31,"15min":0.31},"max":{"1min":1.2370000000000001,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":0.60899999999999999}]},{"osd":3,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84799999999999998,"15min":0.86099999999999999},"min":{"1min":0.56499999999999995,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.101,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0209999999999999,"15min":0.95999999999999996},"min":{"1min":0.45200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.149,"5min":13.539,"15min":13.539},"last":0.97499999999999998}]},{"osd":4,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.89600000000000002,"15min":0.92900000000000005},"min":{"1min":0.60199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.242,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88500000000000001,"15min":0.86699999999999999},"min":{"1min":0.501,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.204,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.038}]},{"osd":5,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.101,"15min":1.2310000000000001},"min":{"1min":0.66700000000000004,"5min":0.312,"15min":0.312},"max":{"1min":1.1970000000000001,"5min":11.922000000000001,"15min":11.922000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1,"15min":0.995},"min":{"1min":0.46999999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3080000000000001,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":1.232}]},{"osd":7,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0269999999999999,"15min":0.995},"min":{"1min":0.60599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.353,"5min":2.129,"15min":2.129},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.101,"15min":1.101},"min":{"1min":0.67300000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3879999999999999,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.006}]}]},{"osd":5,"up_from":14,"seq":60129542204,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1090864,"kb_used_data":843236,"kb_used_omap":1,"kb_used_meta":247614,"kb_avail":93280976,"statfs":{"total":96636764160,"available":95519719424,"internally_reserved":0,"allocated":863473664,"data_stored":869120975,"data_compressed":21037680,"data_compressed_allocated":21954560,"data_compressed_original":28217344,"omap_allocated":1594,"internal_metadata":253557190},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76400000000000001,"15min":0.68999999999999995},"min":{"1min":0.443,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.3919999999999999,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.875,"15min":0.91000000000000003},"min":{"1min":0.29299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.3300000000000001,"5min":4.6390000000000002,"15min":4.6390000000000002},"last":0.749}]},{"osd":1,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.78900000000000003,"15min":0.71399999999999997},"min":{"1min":0.35799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":1.3500000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":1.0049999999999999,"15min":1.181},"min":{"1min":0.376,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.0449999999999999,"5min":8.202,"15min":8.202},"last":0.68700000000000006}]},{"osd":2,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.83299999999999996,"15min":0.78800000000000003},"min":{"1min":0.39900000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4299999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.90300000000000002,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.474,"5min":1.673,"15min":1.673},"last":1.0029999999999999}]},{"osd":3,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":2.0110000000000001,"15min":2.8210000000000002},"min":{"1min":0.33600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.423,"5min":44.186999999999998,"15min":44.186999999999998},"last":1.153},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.081,"15min":1.198},"min":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1419999999999999,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.86199999999999999}]},{"osd":4,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.1659999999999999,"15min":1.3240000000000001},"min":{"1min":0.29899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.3520000000000001,"5min":11.438000000000001,"15min":11.438000000000001},"last":1.306},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.98599999999999999,"15min":0.95999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.171,"5min":2.181,"15min":2.181},"last":0.79300000000000004}]},{"osd":6,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":0.93500000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1830000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":1.097,"15min":1.335},"min":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.222,"5min":11.474,"15min":11.474},"last":1.0569999999999999}]},{"osd":7,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.107,"15min":1.081},"min":{"1min":0.38200000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.542,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.01,"5min":1.1100000000000001,"15min":1.1579999999999999},"min":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3009999999999999,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.1180000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542205,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1920756,"kb_used_data":1416424,"kb_used_omap":1,"kb_used_meta":504318,"kb_avail":92451084,"statfs":{"total":96636764160,"available":94669910016,"internally_reserved":0,"allocated":1450418176,"data_stored":1463127323,"data_compressed":51243088,"data_compressed_allocated":53432320,"data_compressed_original":67096576,"omap_allocated":1594,"internal_metadata":516422086},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":0.91800000000000004,"15min":0.75800000000000001},"min":{"1min":0.28499999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":12.618,"5min":12.618,"15min":12.618},"last":1.327},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.0409999999999999,"15min":0.92000000000000004},"min":{"1min":0.35699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":12.548,"5min":12.548,"15min":12.548},"last":1.0469999999999999}]},{"osd":1,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.751,"15min":0.755},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.1970000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.5509999999999999,"5min":0.96099999999999997,"15min":0.81799999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":12.449999999999999,"5min":12.449999999999999,"15min":12.449999999999999},"last":1.1279999999999999}]},{"osd":2,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.33900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.097,"5min":2.226,"15min":2.226},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.048,"15min":0.92800000000000005},"min":{"1min":0.42899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":12.673,"5min":12.673,"15min":12.673},"last":0.84399999999999997}]},{"osd":3,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.90400000000000003,"15min":0.88400000000000001},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.2070000000000001,"5min":3.5329999999999999,"15min":3.5329999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.41899999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.335,"5min":2.254,"15min":2.254},"last":1.073}]},{"osd":5,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.1040000000000001,"15min":1.0349999999999999},"min":{"1min":0.40200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":12.721,"5min":12.721,"15min":12.721},"last":1.252},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.95999999999999996},"min":{"1min":0.39500000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.369,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.1990000000000001}]},{"osd":6,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.1919999999999999,"15min":1.04},"min":{"1min":0.55600000000000005,"5min":0.312,"15min":0.312},"max":{"1min":12.765000000000001,"5min":12.765000000000001,"15min":12.765000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0860000000000001,"15min":1.087},"min":{"1min":0.57699999999999996,"5min":0.372,"15min":0.372},"max":{"1min":1.431,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.3859999999999999}]},{"osd":7,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.038,"15min":1.0569999999999999},"min":{"1min":0.51800000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.456,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.091,"15min":1.0629999999999999},"min":{"1min":0.505,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.6120000000000001,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.1659999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1378076,"kb_used_data":1018004,"kb_used_omap":1,"kb_used_meta":360062,"kb_avail":92993764,"statfs":{"total":96636764160,"available":95225614336,"internally_reserved":0,"allocated":1042436096,"data_stored":1047505319,"data_compressed":29839922,"data_compressed_allocated":30982144,"data_compressed_original":36835328,"omap_allocated":1593,"internal_metadata":368703943},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60599999999999998,"15min":0.55000000000000004},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.792,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.74199999999999999,"15min":0.80600000000000005},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1.4770000000000001,"5min":4.7489999999999997,"15min":4.7489999999999997},"last":0.312}]},{"osd":2,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.68700000000000006,"15min":0.625},"min":{"1min":0.28399999999999997,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.90100000000000002},"min":{"1min":0.33200000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.6579999999999999,"5min":4.7880000000000003,"15min":4.7880000000000003},"last":0.26700000000000002}]},{"osd":3,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.5429999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.79400000000000004,"15min":0.76300000000000001},"min":{"1min":0.315,"5min":0.307,"15min":0.307},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.438}]},{"osd":4,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.01,"15min":1.085},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.232,"5min":4.8440000000000003,"15min":4.8440000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.91400000000000003,"15min":0.90400000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.258,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.78000000000000003}]},{"osd":5,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.94799999999999995,"15min":0.93200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.47,"5min":2.722,"15min":2.722},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.0289999999999999,"15min":1.107},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.474,"5min":4.8860000000000001,"15min":4.8860000000000001},"last":0.72799999999999998}]},{"osd":6,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.96199999999999997,"15min":0.88900000000000001},"min":{"1min":0.40000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.5449999999999999,"5min":1.76,"15min":1.76},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.94199999999999995,"15min":0.89300000000000002},"min":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3560000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.57699999999999996}]},{"osd":7,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2629999999999999,"15min":1.4019999999999999},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.4980000000000002,"5min":8.6839999999999993,"15min":8.6839999999999993},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.304,"5min":1.1970000000000001,"15min":1.2509999999999999},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.4649999999999999,"5min":4.4649999999999999,"15min":4.4649999999999999},"last":0.54000000000000004}]}]},{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1329564,"kb_used_data":959696,"kb_used_omap":1,"kb_used_meta":369854,"kb_avail":93042276,"statfs":{"total":96636764160,"available":95275290624,"internally_reserved":0,"allocated":982728704,"data_stored":987914375,"data_compressed":23928438,"data_compressed_allocated":24997888,"data_compressed_original":30998528,"omap_allocated":1589,"internal_metadata":378730955},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":1.3640000000000001,"15min":1.0680000000000001},"min":{"1min":0.45700000000000002,"5min":0.224,"15min":0.224},"max":{"1min":1.1839999999999999,"5min":46.598999999999997,"15min":46.598999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":1.0309999999999999,"15min":1.2709999999999999},"min":{"1min":0.32700000000000001,"5min":0.216,"15min":0.216},"max":{"1min":1.141,"5min":15.144,"15min":15.144},"last":0.53600000000000003}]},{"osd":1,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.80900000000000005,"15min":0.77500000000000002},"min":{"1min":0.35699999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.347,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.018,"5min":0.89700000000000002,"15min":0.82699999999999996},"min":{"1min":0.48599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3300000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.748}]},{"osd":2,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.89500000000000002,"15min":0.871},"min":{"1min":0.33000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.262,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.94899999999999995,"15min":0.873},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4419999999999999,"5min":3.387,"15min":3.387},"last":0.80100000000000005}]},{"osd":3,"last update":"Mon Apr 22 05:54:50 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.999,"15min":0.85899999999999999},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3520000000000001,"5min":7.1699999999999999,"15min":7.1699999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.89600000000000002,"15min":0.82799999999999996},"min":{"1min":0.45600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.399,"5min":3.6299999999999999,"15min":3.6299999999999999},"last":0.98599999999999999}]},{"osd":4,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.258,"15min":1.2909999999999999},"min":{"1min":0.34399999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.4650000000000001,"5min":9.6620000000000008,"15min":9.6620000000000008},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.94799999999999995,"15min":0.90200000000000002},"min":{"1min":0.40600000000000003,"5min":0.252,"15min":0.252},"max":{"1min":1.335,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.0309999999999999}]},{"osd":5,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.95399999999999996,"15min":0.89900000000000002},"min":{"1min":0.29299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.54,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1659999999999999,"15min":0.997},"min":{"1min":0.39100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.72,"5min":7.2430000000000003,"15min":7.2430000000000003},"last":1.121}]},{"osd":6,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.04,"15min":0.95599999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5649999999999999,"5min":3.5939999999999999,"15min":3.5939999999999999},"last":1.143},{"interface":"front","average":{"1min":1.208,"5min":1.2450000000000001,"15min":1.0649999999999999},"min":{"1min":0.53400000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.794,"5min":7.2960000000000003,"15min":7.2960000000000003},"last":1.1619999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1412660,"kb_used_data":1016552,"kb_used_omap":1,"kb_used_meta":396094,"kb_avail":92959180,"statfs":{"total":96636764160,"available":95190200320,"internally_reserved":0,"allocated":1040949248,"data_stored":1044391251,"data_compressed":24230251,"data_compressed_allocated":25128960,"data_compressed_original":29396992,"omap_allocated":1592,"internal_metadata":405600712},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.69199999999999995,"15min":0.82399999999999995},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":1.079,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":1.4970000000000001,"15min":2.2440000000000002},"min":{"1min":0.28799999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.129,"5min":22.721,"15min":22.721},"last":0.32100000000000001}]},{"osd":1,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.80700000000000005,"15min":0.95099999999999996},"min":{"1min":0.23999999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.5840000000000001,"5min":9.6560000000000006,"15min":9.6560000000000006},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.3100000000000001,"5min":2.028,"15min":2.028},"last":0.47099999999999997}]},{"osd":2,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":1.3009999999999999,"15min":1.619},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.3029999999999999,"5min":22.763999999999999,"15min":22.763999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.94299999999999995,"15min":1.0920000000000001},"min":{"1min":0.313,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.6190000000000002,"5min":9.7739999999999991,"15min":9.7739999999999991},"last":0.51200000000000001}]},{"osd":4,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.7270000000000001,"15min":1.8879999999999999},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.657,"5min":27.384,"15min":27.384},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.571,"15min":1.931},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.508,"5min":22.824000000000002,"15min":22.824000000000002},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.762,"15min":2.3220000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.4039999999999999,"5min":25.565999999999999,"15min":25.565999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.98799999999999999,"15min":0.93999999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.6829999999999998,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.81799999999999995}]},{"osd":6,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.6419999999999999,"15min":1.9570000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.706,"5min":22.794,"15min":22.794},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.006,"5min":1.171,"15min":1.323},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6240000000000001,"5min":9.9559999999999995,"15min":9.9559999999999995},"last":0.871}]},{"osd":7,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":2.4089999999999998,"15min":3.1589999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.796,"5min":36.770000000000003,"15min":36.770000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.4039999999999999,"15min":1.4059999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.9100000000000001,"5min":16.239000000000001,"15min":16.239000000000001},"last":0.75800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":989300,"kb_used_data":665384,"kb_used_omap":1,"kb_used_meta":323902,"kb_avail":93382540,"statfs":{"total":96636764160,"available":95623720960,"internally_reserved":0,"allocated":681353216,"data_stored":685339487,"data_compressed":26441385,"data_compressed_allocated":27336704,"data_compressed_original":31993856,"omap_allocated":1594,"internal_metadata":331676102},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.66600000000000004,"15min":0.70699999999999996},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.83099999999999996,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.26200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.1679999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.25900000000000001}]},{"osd":2,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.26100000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.014,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.68600000000000005,"15min":0.61799999999999999},"min":{"1min":0.34599999999999997,"5min":0.22,"15min":0.22},"max":{"1min":1.0900000000000001,"5min":3.871,"15min":3.871},"last":0.39200000000000002}]},{"osd":3,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.29699999999999999,"5min":0.252,"15min":0.252},"max":{"1min":1.1970000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.83499999999999996,"15min":0.85199999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.256,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.29799999999999999}]},{"osd":4,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.0409999999999999,"15min":1.1180000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.242,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.01,"5min":1.044,"15min":1.044},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":2.0880000000000001,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":0.77800000000000002}]},{"osd":5,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.98299999999999998,"15min":0.94499999999999995},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.2650000000000001,"5min":3.8460000000000001,"15min":3.8460000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.98799999999999999,"15min":0.96999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.325,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.64300000000000002}]},{"osd":6,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97699999999999998,"15min":0.92300000000000004},"min":{"1min":0.38400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3999999999999999,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.98799999999999999,"15min":0.95799999999999996},"min":{"1min":0.42699999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.80600000000000005}]},{"osd":7,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.042,"15min":1.004},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.1800000000000002,"5min":3.8159999999999998,"15min":3.8159999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.0720000000000001,"15min":1.0369999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.615,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":0.627}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1042149376,"data_stored":1046890840,"data_compressed":29834343,"data_compressed_allocated":30949376,"data_compressed_original":36323328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":681115648,"data_stored":685252972,"data_compressed":26441385,"data_compressed_allocated":27336704,"data_compressed_original":31993856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417832960,"data_stored":419126404,"data_compressed":12754442,"data_compressed_allocated":13279232,"data_compressed_original":14893056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1040711680,"data_stored":1044304736,"data_compressed":24230251,"data_compressed_allocated":25128960,"data_compressed_original":29396992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1450180608,"data_stored":1463040808,"data_compressed":51243088,"data_compressed_allocated":53432320,"data_compressed_original":67096576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":863236096,"data_stored":869034460,"data_compressed":21037680,"data_compressed_allocated":21954560,"data_compressed_original":28217344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":318418944,"data_stored":318711896,"data_compressed":4811634,"data_compressed_allocated":5001216,"data_compressed_original":5484544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":982437888,"data_stored":987298932,"data_compressed":23922859,"data_compressed_allocated":24965120,"data_compressed_original":30486528,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T05:55:25.237 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.237 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.237 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.237 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.237 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.238 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.238 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.238 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.238 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.238 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.238 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.238 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.238 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-22T05:50:40.020947+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=40, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.238 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.239 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-22T05:50:40.020947+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=40, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.239 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.239 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-22T05:50:40.020947+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=40, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.239 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.239 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-22T05:50:40.020947+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=40, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.239 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.239 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.239 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-22T05:50:40.020947+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=40, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.239 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.239 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-22T05:50:40.020947+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=40, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.240 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.240 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-22T05:50:40.020947+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=40, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.240 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.240 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-22T05:50:40.020947+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=40, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.240 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-22T05:50:39.011105+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=39, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.240 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.240 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.240 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.240 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.241 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.241 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.241 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.241 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.241 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.241 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.241 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:25.241 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-22T05:55:45.243 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T05:55:45.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.378+0000 7f8df4791640 1 Processor -- start 2024-04-22T05:55:45.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.378+0000 7f8df4791640 1 -- start start 2024-04-22T05:55:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df4791640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8dec0b42d0 0x7f8dec13e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df4791640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8dec1432d0 0x7f8dec13eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df4791640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dec13f0a0 con 0x7f8dec0a8ab0 2024-04-22T05:55:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df4791640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dec0b46b0 con 0x7f8dec1432d0 2024-04-22T05:55:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df4791640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec0b4ea0 con 0x7f8dec0b42d0 2024-04-22T05:55:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df252d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8dec0b42d0 0x7f8dec13e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df1d2c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8dec0a8ab0 0x7f8dec142bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45860/0 (socket says 172.21.15.40:45860) 2024-04-22T05:55:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df1d2c640 1 -- 172.21.15.40:0/1229368785 learned_addr learned my addr 172.21.15.40:0/1229368785 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:55:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df2d2e640 1 --2- 172.21.15.40:0/1229368785 >> v2:172.21.15.62:3300/0 conn(0x7f8dec1432d0 0x7f8dec13eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df252d640 1 -- 172.21.15.40:0/1229368785 >> v2:172.21.15.62:3300/0 conn(0x7f8dec1432d0 msgr2=0x7f8dec13eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:55:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df252d640 1 --2- 172.21.15.40:0/1229368785 >> v2:172.21.15.62:3300/0 conn(0x7f8dec1432d0 0x7f8dec13eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df252d640 1 -- 172.21.15.40:0/1229368785 >> v1:172.21.15.40:6789/0 conn(0x7f8dec0a8ab0 legacy=0x7f8dec142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df252d640 1 -- 172.21.15.40:0/1229368785 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec143720 con 0x7f8dec0b42d0 2024-04-22T05:55:45.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df252d640 1 --2- 172.21.15.40:0/1229368785 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8dec0b42d0 0x7f8dec13e620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009d90 tx=0x7f8de401c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75184e76492f121c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:45.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df152b640 1 -- 172.21.15.40:0/1229368785 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de40a66f0 con 0x7f8dec0b42d0 2024-04-22T05:55:45.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df152b640 1 -- 172.21.15.40:0/1229368785 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de40a6850 con 0x7f8dec0b42d0 2024-04-22T05:55:45.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.382+0000 7f8df152b640 1 -- 172.21.15.40:0/1229368785 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de40a6be0 con 0x7f8dec0b42d0 2024-04-22T05:55:45.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 -- 172.21.15.40:0/1229368785 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8dec0b42d0 msgr2=0x7f8dec13e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:45.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 --2- 172.21.15.40:0/1229368785 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8dec0b42d0 0x7f8dec13e620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009d90 tx=0x7f8de401c720 comp rx=0 tx=0).stop 2024-04-22T05:55:45.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 -- 172.21.15.40:0/1229368785 shutdown_connections 2024-04-22T05:55:45.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 --2- 172.21.15.40:0/1229368785 >> v2:172.21.15.62:3300/0 conn(0x7f8dec1432d0 0x7f8dec13eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:45.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 --2- 172.21.15.40:0/1229368785 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8dec0b42d0 0x7f8dec13e620 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:45.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 -- 172.21.15.40:0/1229368785 >> 172.21.15.40:0/1229368785 conn(0x7f8dec0b1c60 msgr2=0x7f8dec0b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:45.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 -- 172.21.15.40:0/1229368785 shutdown_connections 2024-04-22T05:55:45.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 -- 172.21.15.40:0/1229368785 wait complete. 2024-04-22T05:55:45.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 Processor -- start 2024-04-22T05:55:45.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 -- start start 2024-04-22T05:55:45.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8dec0a8ab0 0x7f8dec132190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:45.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dec0b42d0 0x7f8dec13c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:45.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dec1432d0 0x7f8dec13c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:45.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec12c210 con 0x7f8dec1432d0 2024-04-22T05:55:45.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec12c6a0 con 0x7f8dec0b42d0 2024-04-22T05:55:45.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec13cc30 con 0x7f8dec0a8ab0 2024-04-22T05:55:45.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df252d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8dec0a8ab0 0x7f8dec132190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:45.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df252d640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8dec0a8ab0 0x7f8dec132190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:60230/0 (socket says 172.21.15.40:60230) 2024-04-22T05:55:45.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df252d640 1 -- 172.21.15.40:0/3429970076 learned_addr learned my addr 172.21.15.40:0/3429970076 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:55:45.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df2d2e640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dec1432d0 0x7f8dec13c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:45.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df1d2c640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dec0b42d0 0x7f8dec13c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:45.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df252d640 1 -- 172.21.15.40:0/3429970076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dec0b42d0 msgr2=0x7f8dec13c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:45.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df252d640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dec0b42d0 0x7f8dec13c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:45.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df252d640 1 -- 172.21.15.40:0/3429970076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dec1432d0 msgr2=0x7f8dec13c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:45.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df252d640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dec1432d0 0x7f8dec13c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:45.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df252d640 1 -- 172.21.15.40:0/3429970076 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de4009a70 con 0x7f8dec0a8ab0 2024-04-22T05:55:45.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df252d640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8dec0a8ab0 0x7f8dec132190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8de40b0040 tx=0x7f8de40194c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:45.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8deaffd640 1 -- 172.21.15.40:0/3429970076 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de40b8840 con 0x7f8dec0a8ab0 2024-04-22T05:55:45.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8deaffd640 1 -- 172.21.15.40:0/3429970076 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de40b89c0 con 0x7f8dec0a8ab0 2024-04-22T05:55:45.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 -- 172.21.15.40:0/3429970076 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dec13a5e0 con 0x7f8dec0a8ab0 2024-04-22T05:55:45.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8deaffd640 1 -- 172.21.15.40:0/3429970076 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de40b8d70 con 0x7f8dec0a8ab0 2024-04-22T05:55:45.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.386+0000 7f8df4791640 1 -- 172.21.15.40:0/3429970076 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dec13dd50 con 0x7f8dec0a8ab0 2024-04-22T05:55:45.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.390+0000 7f8df4791640 1 -- 172.21.15.40:0/3429970076 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dec04e690 con 0x7f8dec0a8ab0 2024-04-22T05:55:45.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.390+0000 7f8deaffd640 1 -- 172.21.15.40:0/3429970076 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8de40ae420 con 0x7f8dec0a8ab0 2024-04-22T05:55:45.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.390+0000 7f8deaffd640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8dd8066690 0x7f8dd8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:55:45.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.390+0000 7f8deaffd640 1 -- 172.21.15.40:0/3429970076 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f8de40aa070 con 0x7f8dec0a8ab0 2024-04-22T05:55:45.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.390+0000 7f8df1d2c640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8dd8066690 0x7f8dd8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:55:45.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.390+0000 7f8df1d2c640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8dd8066690 0x7f8dd8068b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8dec12df40 tx=0x7f8de0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:55:45.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.394+0000 7f8deaffd640 1 -- 172.21.15.40:0/3429970076 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f8de40ecd20 con 0x7f8dec0a8ab0 2024-04-22T05:55:45.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.550+0000 7f8df4791640 1 -- 172.21.15.40:0/3429970076 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8dec12d0a0 con 0x7f8dd8066690 2024-04-22T05:55:45.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.566+0000 7f8deaffd640 1 -- 172.21.15.40:0/3429970076 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+153954 (secure 0 0 0) 0x7f8de0007070 con 0x7f8dd8066690 2024-04-22T05:55:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:55:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-22T05:55:45.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.566+0000 7f8df4791640 1 -- 172.21.15.40:0/3429970076 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8dd8066690 msgr2=0x7f8dd8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.566+0000 7f8df4791640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8dd8066690 0x7f8dd8068b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8dec12df40 tx=0x7f8de0009040 comp rx=0 tx=0).stop 2024-04-22T05:55:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.566+0000 7f8df4791640 1 -- 172.21.15.40:0/3429970076 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8dec0a8ab0 msgr2=0x7f8dec132190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:55:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.566+0000 7f8df4791640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8dec0a8ab0 0x7f8dec132190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8de40b0040 tx=0x7f8de40194c0 comp rx=0 tx=0).stop 2024-04-22T05:55:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.566+0000 7f8df4791640 1 -- 172.21.15.40:0/3429970076 shutdown_connections 2024-04-22T05:55:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.566+0000 7f8df4791640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f8dd8066690 0x7f8dd8068b50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.566+0000 7f8df4791640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dec0b42d0 0x7f8dec13c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.566+0000 7f8df4791640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dec1432d0 0x7f8dec13c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.566+0000 7f8df4791640 1 --2- 172.21.15.40:0/3429970076 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f8dec0a8ab0 0x7f8dec132190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:55:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.566+0000 7f8df4791640 1 -- 172.21.15.40:0/3429970076 >> 172.21.15.40:0/3429970076 conn(0x7f8dec0b1c60 msgr2=0x7f8dec13b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:55:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.566+0000 7f8df4791640 1 -- 172.21.15.40:0/3429970076 shutdown_connections 2024-04-22T05:55:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:55:45.570+0000 7f8df4791640 1 -- 172.21.15.40:0/3429970076 wait complete. 2024-04-22T05:55:45.588 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-04-22T05:55:45.033603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3568096024,"num_objects":879,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":879,"num_whiteouts":0,"num_read":2676,"num_read_kb":2676,"num_write":157303,"num_write_kb":5628657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159363,"ondisk_log_size":159363,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9135844,"kb_used_data":6638756,"kb_used_omap":12,"kb_used_meta":2497011,"kb_avail":745838876,"statfs":{"total":773094113280,"available":763739009024,"internally_reserved":0,"allocated":6798086144,"data_stored":6835410060,"data_compressed":194286840,"data_compressed_allocated":202113024,"data_compressed_original":244916224,"omap_allocated":12744,"internal_metadata":2556939832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003957"},"pg_stats":[{"pgid":"3.1f","version":"23'3351","reported_seq":3392,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:37.413059+0000","last_change":"2024-04-22T05:55:37.413304+0000","last_active":"2024-04-22T05:55:37.413059+0000","last_peered":"2024-04-22T05:55:37.413059+0000","last_clean":"2024-04-22T05:55:37.413059+0000","last_became_active":"2024-04-22T05:50:47.120501+0000","last_became_peered":"2024-04-22T05:50:47.120501+0000","last_unstale":"2024-04-22T05:55:37.413059+0000","last_undegraded":"2024-04-22T05:55:37.413059+0000","last_fullsized":"2024-04-22T05:55:37.413059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3351","last_scrub_stamp":"2024-04-22T05:55:37.412994+0000","last_deep_scrub":"23'3351","last_deep_scrub_stamp":"2024-04-22T05:55:37.412994+0000","last_clean_scrub_stamp":"2024-04-22T05:55:37.412994+0000","objects_scrubbed":29,"log_size":3351,"log_dups_size":0,"ondisk_log_size":3351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:04:03.177160+0000","scrub_duration":0.41623528900000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109323296,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":3347,"num_write_kb":168857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'5529","reported_seq":5565,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:35.652551+0000","last_change":"2024-04-22T05:55:35.652789+0000","last_active":"2024-04-22T05:55:35.652551+0000","last_peered":"2024-04-22T05:55:35.652551+0000","last_clean":"2024-04-22T05:55:35.652551+0000","last_became_active":"2024-04-22T05:50:47.117274+0000","last_became_peered":"2024-04-22T05:50:47.117274+0000","last_unstale":"2024-04-22T05:55:35.652551+0000","last_undegraded":"2024-04-22T05:55:35.652551+0000","last_fullsized":"2024-04-22T05:55:35.652551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5529","last_scrub_stamp":"2024-04-22T05:55:35.652500+0000","last_deep_scrub":"23'5529","last_deep_scrub_stamp":"2024-04-22T05:55:35.652500+0000","last_clean_scrub_stamp":"2024-04-22T05:55:35.652500+0000","objects_scrubbed":32,"log_size":5529,"log_dups_size":0,"ondisk_log_size":5529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:24.731938+0000","scrub_duration":0.43440045300000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125960704,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":28,"num_read_kb":84,"num_write":5488,"num_write_kb":185156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"23'3418","reported_seq":3443,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.201673+0000","last_change":"2024-04-22T05:50:47.122061+0000","last_active":"2024-04-22T05:55:18.201673+0000","last_peered":"2024-04-22T05:55:18.201673+0000","last_clean":"2024-04-22T05:55:18.201673+0000","last_became_active":"2024-04-22T05:50:47.120601+0000","last_became_peered":"2024-04-22T05:50:47.120601+0000","last_unstale":"2024-04-22T05:55:18.201673+0000","last_undegraded":"2024-04-22T05:55:18.201673+0000","last_fullsized":"2024-04-22T05:55:18.201673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3418,"log_dups_size":0,"ondisk_log_size":3418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72812200,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":3406,"num_write_kb":94339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2981","reported_seq":2995,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.185853+0000","last_change":"2024-04-22T05:50:47.117092+0000","last_active":"2024-04-22T05:55:18.185853+0000","last_peered":"2024-04-22T05:55:18.185853+0000","last_clean":"2024-04-22T05:55:18.185853+0000","last_became_active":"2024-04-22T05:50:47.115803+0000","last_became_peered":"2024-04-22T05:50:47.115803+0000","last_unstale":"2024-04-22T05:55:18.185853+0000","last_undegraded":"2024-04-22T05:55:18.185853+0000","last_fullsized":"2024-04-22T05:55:18.185853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":2981,"log_dups_size":0,"ondisk_log_size":2981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89522176,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2976,"num_write_kb":149441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"23'5258","reported_seq":5309,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:24.558832+0000","last_change":"2024-04-22T05:55:24.558948+0000","last_active":"2024-04-22T05:55:24.558832+0000","last_peered":"2024-04-22T05:55:24.558832+0000","last_clean":"2024-04-22T05:55:24.558832+0000","last_became_active":"2024-04-22T05:50:47.123073+0000","last_became_peered":"2024-04-22T05:50:47.123073+0000","last_unstale":"2024-04-22T05:55:24.558832+0000","last_undegraded":"2024-04-22T05:55:24.558832+0000","last_fullsized":"2024-04-22T05:55:24.558832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5258","last_scrub_stamp":"2024-04-22T05:55:24.558811+0000","last_deep_scrub":"23'5258","last_deep_scrub_stamp":"2024-04-22T05:55:24.558811+0000","last_clean_scrub_stamp":"2024-04-22T05:55:24.558811+0000","objects_scrubbed":36,"log_size":5258,"log_dups_size":0,"ondisk_log_size":5258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:50:19.675491+0000","scrub_duration":0.50381928200000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150994944,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":90,"num_read_kb":78,"num_write":5209,"num_write_kb":201924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"23'6531","reported_seq":6566,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:30.328645+0000","last_change":"2024-04-22T05:55:30.328890+0000","last_active":"2024-04-22T05:55:30.328645+0000","last_peered":"2024-04-22T05:55:30.328645+0000","last_clean":"2024-04-22T05:55:30.328645+0000","last_became_active":"2024-04-22T05:50:47.118038+0000","last_became_peered":"2024-04-22T05:50:47.118038+0000","last_unstale":"2024-04-22T05:55:30.328645+0000","last_undegraded":"2024-04-22T05:55:30.328645+0000","last_fullsized":"2024-04-22T05:55:30.328645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6531","last_scrub_stamp":"2024-04-22T05:55:30.328593+0000","last_deep_scrub":"23'6531","last_deep_scrub_stamp":"2024-04-22T05:55:30.328593+0000","last_clean_scrub_stamp":"2024-04-22T05:55:30.328593+0000","objects_scrubbed":28,"log_size":6531,"log_dups_size":0,"ondisk_log_size":6531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:22.808632+0000","scrub_duration":0.56155419200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113311744,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":6483,"num_write_kb":139343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"23'4782","reported_seq":4810,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:42.670477+0000","last_change":"2024-04-22T05:55:42.670599+0000","last_active":"2024-04-22T05:55:42.670477+0000","last_peered":"2024-04-22T05:55:42.670477+0000","last_clean":"2024-04-22T05:55:42.670477+0000","last_became_active":"2024-04-22T05:50:47.113368+0000","last_became_peered":"2024-04-22T05:50:47.113368+0000","last_unstale":"2024-04-22T05:55:42.670477+0000","last_undegraded":"2024-04-22T05:55:42.670477+0000","last_fullsized":"2024-04-22T05:55:42.670477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4782","last_scrub_stamp":"2024-04-22T05:55:42.670447+0000","last_deep_scrub":"23'4782","last_deep_scrub_stamp":"2024-04-22T05:55:42.670447+0000","last_clean_scrub_stamp":"2024-04-22T05:55:42.670447+0000","objects_scrubbed":27,"log_size":4782,"log_dups_size":0,"ondisk_log_size":4782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:19:50.199229+0000","scrub_duration":0.46783407300000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112566272,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":123245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"23'8079","reported_seq":8114,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:40.814520+0000","last_change":"2024-04-22T05:55:40.814693+0000","last_active":"2024-04-22T05:55:40.814520+0000","last_peered":"2024-04-22T05:55:40.814520+0000","last_clean":"2024-04-22T05:55:40.814520+0000","last_became_active":"2024-04-22T05:50:47.113569+0000","last_became_peered":"2024-04-22T05:50:47.113569+0000","last_unstale":"2024-04-22T05:55:40.814520+0000","last_undegraded":"2024-04-22T05:55:40.814520+0000","last_fullsized":"2024-04-22T05:55:40.814520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8079","last_scrub_stamp":"2024-04-22T05:55:40.814475+0000","last_deep_scrub":"23'8079","last_deep_scrub_stamp":"2024-04-22T05:55:40.814475+0000","last_clean_scrub_stamp":"2024-04-22T05:55:40.814475+0000","objects_scrubbed":26,"log_size":8079,"log_dups_size":0,"ondisk_log_size":8079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:13:10.549067+0000","scrub_duration":0.57767474900000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":8065,"num_write_kb":185808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"23'6498","reported_seq":6537,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:34.396028+0000","last_change":"2024-04-22T05:55:34.396222+0000","last_active":"2024-04-22T05:55:34.396028+0000","last_peered":"2024-04-22T05:55:34.396028+0000","last_clean":"2024-04-22T05:55:34.396028+0000","last_became_active":"2024-04-22T05:50:47.121332+0000","last_became_peered":"2024-04-22T05:50:47.121332+0000","last_unstale":"2024-04-22T05:55:34.396028+0000","last_undegraded":"2024-04-22T05:55:34.396028+0000","last_fullsized":"2024-04-22T05:55:34.396028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6498","last_scrub_stamp":"2024-04-22T05:55:34.395983+0000","last_deep_scrub":"23'6498","last_deep_scrub_stamp":"2024-04-22T05:55:34.395983+0000","last_clean_scrub_stamp":"2024-04-22T05:55:34.395983+0000","objects_scrubbed":27,"log_size":6498,"log_dups_size":0,"ondisk_log_size":6498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:07.124227+0000","scrub_duration":0.428001927,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111009792,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":6495,"num_write_kb":211703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"23'3762","reported_seq":3779,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:23.952169+0000","last_change":"2024-04-22T05:50:47.120024+0000","last_active":"2024-04-22T05:55:23.952169+0000","last_peered":"2024-04-22T05:55:23.952169+0000","last_clean":"2024-04-22T05:55:23.952169+0000","last_became_active":"2024-04-22T05:50:47.119031+0000","last_became_peered":"2024-04-22T05:50:47.119031+0000","last_unstale":"2024-04-22T05:55:23.952169+0000","last_undegraded":"2024-04-22T05:55:23.952169+0000","last_fullsized":"2024-04-22T05:55:23.952169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3762,"log_dups_size":0,"ondisk_log_size":3762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":3747,"num_write_kb":173061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"22'7962","reported_seq":7977,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:24.893905+0000","last_change":"2024-04-22T05:50:47.119142+0000","last_active":"2024-04-22T05:55:24.893905+0000","last_peered":"2024-04-22T05:55:24.893905+0000","last_clean":"2024-04-22T05:55:24.893905+0000","last_became_active":"2024-04-22T05:50:47.118399+0000","last_became_peered":"2024-04-22T05:50:47.118399+0000","last_unstale":"2024-04-22T05:55:24.893905+0000","last_undegraded":"2024-04-22T05:55:24.893905+0000","last_fullsized":"2024-04-22T05:55:24.893905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":7962,"log_dups_size":0,"ondisk_log_size":7962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7958,"num_write_kb":234882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"23'7086","reported_seq":7143,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:28.452359+0000","last_change":"2024-04-22T05:55:28.452359+0000","last_active":"2024-04-22T05:55:28.452359+0000","last_peered":"2024-04-22T05:55:28.452359+0000","last_clean":"2024-04-22T05:55:28.452359+0000","last_became_active":"2024-04-22T05:50:47.130842+0000","last_became_peered":"2024-04-22T05:50:47.130842+0000","last_unstale":"2024-04-22T05:55:28.452359+0000","last_undegraded":"2024-04-22T05:55:28.452359+0000","last_fullsized":"2024-04-22T05:55:28.452359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7086","last_scrub_stamp":"2024-04-22T05:55:28.452110+0000","last_deep_scrub":"23'7086","last_deep_scrub_stamp":"2024-04-22T05:55:28.452110+0000","last_clean_scrub_stamp":"2024-04-22T05:55:28.452110+0000","objects_scrubbed":38,"log_size":7086,"log_dups_size":0,"ondisk_log_size":7086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:22:37.268459+0000","scrub_duration":0.68650581600000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149815296,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":118,"num_read_kb":94,"num_write":7055,"num_write_kb":232332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:28.855161+0000","last_change":"2024-04-22T05:55:28.855464+0000","last_active":"2024-04-22T05:55:28.855161+0000","last_peered":"2024-04-22T05:55:28.855161+0000","last_clean":"2024-04-22T05:55:28.855161+0000","last_became_active":"2024-04-22T05:50:41.049696+0000","last_became_peered":"2024-04-22T05:50:41.049696+0000","last_unstale":"2024-04-22T05:55:28.855161+0000","last_undegraded":"2024-04-22T05:55:28.855161+0000","last_fullsized":"2024-04-22T05:55:28.855161+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:28.855062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:28.855062+0000","last_clean_scrub_stamp":"2024-04-22T05:55:28.855062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:51:54.355467+0000","scrub_duration":0.003271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'4474","reported_seq":4522,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:26.512866+0000","last_change":"2024-04-22T05:55:26.513066+0000","last_active":"2024-04-22T05:55:26.512866+0000","last_peered":"2024-04-22T05:55:26.512866+0000","last_clean":"2024-04-22T05:55:26.512866+0000","last_became_active":"2024-04-22T05:50:47.113838+0000","last_became_peered":"2024-04-22T05:50:47.113838+0000","last_unstale":"2024-04-22T05:55:26.512866+0000","last_undegraded":"2024-04-22T05:55:26.512866+0000","last_fullsized":"2024-04-22T05:55:26.512866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4474","last_scrub_stamp":"2024-04-22T05:55:26.512811+0000","last_deep_scrub":"23'4474","last_deep_scrub_stamp":"2024-04-22T05:55:26.512811+0000","last_clean_scrub_stamp":"2024-04-22T05:55:26.512811+0000","objects_scrubbed":30,"log_size":4474,"log_dups_size":0,"ondisk_log_size":4474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:29:16.427953+0000","scrub_duration":0.50539391899999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120194896,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":4449,"num_write_kb":155490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:31.181271+0000","last_change":"2024-04-22T05:55:31.181457+0000","last_active":"2024-04-22T05:55:31.181271+0000","last_peered":"2024-04-22T05:55:31.181271+0000","last_clean":"2024-04-22T05:55:31.181271+0000","last_became_active":"2024-04-22T05:50:41.048628+0000","last_became_peered":"2024-04-22T05:50:41.048628+0000","last_unstale":"2024-04-22T05:55:31.181271+0000","last_undegraded":"2024-04-22T05:55:31.181271+0000","last_fullsized":"2024-04-22T05:55:31.181271+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:31.181223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:31.181223+0000","last_clean_scrub_stamp":"2024-04-22T05:55:31.181223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:49:42.189479+0000","scrub_duration":0.0030718529999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'3274","reported_seq":3302,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:33.534972+0000","last_change":"2024-04-22T05:55:33.535217+0000","last_active":"2024-04-22T05:55:33.534972+0000","last_peered":"2024-04-22T05:55:33.534972+0000","last_clean":"2024-04-22T05:55:33.534972+0000","last_became_active":"2024-04-22T05:50:47.129322+0000","last_became_peered":"2024-04-22T05:50:47.129322+0000","last_unstale":"2024-04-22T05:55:33.534972+0000","last_undegraded":"2024-04-22T05:55:33.534972+0000","last_fullsized":"2024-04-22T05:55:33.534972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3274","last_scrub_stamp":"2024-04-22T05:55:33.534920+0000","last_deep_scrub":"23'3274","last_deep_scrub_stamp":"2024-04-22T05:55:33.534920+0000","last_clean_scrub_stamp":"2024-04-22T05:55:33.534920+0000","objects_scrubbed":20,"log_size":3274,"log_dups_size":0,"ondisk_log_size":3274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:52.910713+0000","scrub_duration":0.28762188999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":3214,"num_write_kb":155139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:35.946947+0000","last_change":"2024-04-22T05:55:35.947194+0000","last_active":"2024-04-22T05:55:35.946947+0000","last_peered":"2024-04-22T05:55:35.946947+0000","last_clean":"2024-04-22T05:55:35.946947+0000","last_became_active":"2024-04-22T05:50:41.340643+0000","last_became_peered":"2024-04-22T05:50:41.340643+0000","last_unstale":"2024-04-22T05:55:35.946947+0000","last_undegraded":"2024-04-22T05:55:35.946947+0000","last_fullsized":"2024-04-22T05:55:35.946947+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:35.946899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:35.946899+0000","last_clean_scrub_stamp":"2024-04-22T05:55:35.946899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:08:49.875131+0000","scrub_duration":0.0037993990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'6389","reported_seq":6423,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:39.727480+0000","last_change":"2024-04-22T05:55:39.727688+0000","last_active":"2024-04-22T05:55:39.727480+0000","last_peered":"2024-04-22T05:55:39.727480+0000","last_clean":"2024-04-22T05:55:39.727480+0000","last_became_active":"2024-04-22T05:50:47.172121+0000","last_became_peered":"2024-04-22T05:50:47.172121+0000","last_unstale":"2024-04-22T05:55:39.727480+0000","last_undegraded":"2024-04-22T05:55:39.727480+0000","last_fullsized":"2024-04-22T05:55:39.727480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6389","last_scrub_stamp":"2024-04-22T05:55:39.727424+0000","last_deep_scrub":"23'6389","last_deep_scrub_stamp":"2024-04-22T05:55:39.727424+0000","last_clean_scrub_stamp":"2024-04-22T05:55:39.727424+0000","objects_scrubbed":28,"log_size":6389,"log_dups_size":0,"ondisk_log_size":6389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:56:50.890766+0000","scrub_duration":0.53035004900000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":26,"num_read_kb":18,"num_write":6371,"num_write_kb":148611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:23.316432+0000","last_change":"2024-04-22T05:55:23.316534+0000","last_active":"2024-04-22T05:55:23.316432+0000","last_peered":"2024-04-22T05:55:23.316432+0000","last_clean":"2024-04-22T05:55:23.316432+0000","last_became_active":"2024-04-22T05:50:41.341254+0000","last_became_peered":"2024-04-22T05:50:41.341254+0000","last_unstale":"2024-04-22T05:55:23.316432+0000","last_undegraded":"2024-04-22T05:55:23.316432+0000","last_fullsized":"2024-04-22T05:55:23.316432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:23.316401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:23.316401+0000","last_clean_scrub_stamp":"2024-04-22T05:55:23.316401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:08:51.088728+0000","scrub_duration":0.00144433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'3329","reported_seq":3366,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:42.791313+0000","last_change":"2024-04-22T05:55:42.791521+0000","last_active":"2024-04-22T05:55:42.791313+0000","last_peered":"2024-04-22T05:55:42.791313+0000","last_clean":"2024-04-22T05:55:42.791313+0000","last_became_active":"2024-04-22T05:50:47.120072+0000","last_became_peered":"2024-04-22T05:50:47.120072+0000","last_unstale":"2024-04-22T05:55:42.791313+0000","last_undegraded":"2024-04-22T05:55:42.791313+0000","last_fullsized":"2024-04-22T05:55:42.791313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3329","last_scrub_stamp":"2024-04-22T05:55:42.791269+0000","last_deep_scrub":"23'3329","last_deep_scrub_stamp":"2024-04-22T05:55:42.791269+0000","last_clean_scrub_stamp":"2024-04-22T05:55:42.791269+0000","objects_scrubbed":24,"log_size":3329,"log_dups_size":0,"ondisk_log_size":3329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:33.469332+0000","scrub_duration":0.32450623299999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99223376,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":3325,"num_write_kb":171289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'3657","reported_seq":3694,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:36.200776+0000","last_change":"2024-04-22T05:55:36.201048+0000","last_active":"2024-04-22T05:55:36.200776+0000","last_peered":"2024-04-22T05:55:36.200776+0000","last_clean":"2024-04-22T05:55:36.200776+0000","last_became_active":"2024-04-22T05:50:47.124841+0000","last_became_peered":"2024-04-22T05:50:47.124841+0000","last_unstale":"2024-04-22T05:55:36.200776+0000","last_undegraded":"2024-04-22T05:55:36.200776+0000","last_fullsized":"2024-04-22T05:55:36.200776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3657","last_scrub_stamp":"2024-04-22T05:55:36.200720+0000","last_deep_scrub":"23'3657","last_deep_scrub_stamp":"2024-04-22T05:55:36.200720+0000","last_clean_scrub_stamp":"2024-04-22T05:55:36.200720+0000","objects_scrubbed":28,"log_size":3657,"log_dups_size":0,"ondisk_log_size":3657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:46.227043+0000","scrub_duration":0.37396192499999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":3649,"num_write_kb":178556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:25.655509+0000","last_change":"2024-04-22T05:55:25.655709+0000","last_active":"2024-04-22T05:55:25.655509+0000","last_peered":"2024-04-22T05:55:25.655509+0000","last_clean":"2024-04-22T05:55:25.655509+0000","last_became_active":"2024-04-22T05:50:41.050711+0000","last_became_peered":"2024-04-22T05:50:41.050711+0000","last_unstale":"2024-04-22T05:55:25.655509+0000","last_undegraded":"2024-04-22T05:55:25.655509+0000","last_fullsized":"2024-04-22T05:55:25.655509+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:25.655461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:25.655461+0000","last_clean_scrub_stamp":"2024-04-22T05:55:25.655461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:14:28.626354+0000","scrub_duration":0.0034954550000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"23'5425","reported_seq":5456,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:37.732937+0000","last_change":"2024-04-22T05:55:37.733151+0000","last_active":"2024-04-22T05:55:37.732937+0000","last_peered":"2024-04-22T05:55:37.732937+0000","last_clean":"2024-04-22T05:55:37.732937+0000","last_became_active":"2024-04-22T05:50:47.172218+0000","last_became_peered":"2024-04-22T05:50:47.172218+0000","last_unstale":"2024-04-22T05:55:37.732937+0000","last_undegraded":"2024-04-22T05:55:37.732937+0000","last_fullsized":"2024-04-22T05:55:37.732937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5425","last_scrub_stamp":"2024-04-22T05:55:37.732870+0000","last_deep_scrub":"23'5425","last_deep_scrub_stamp":"2024-04-22T05:55:37.732870+0000","last_clean_scrub_stamp":"2024-04-22T05:55:37.732870+0000","objects_scrubbed":35,"log_size":5425,"log_dups_size":0,"ondisk_log_size":5425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:13:05.798101+0000","scrub_duration":0.59605697599999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138609688,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":5358,"num_write_kb":177059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:22.680694+0000","last_change":"2024-04-22T05:55:22.680782+0000","last_active":"2024-04-22T05:55:22.680694+0000","last_peered":"2024-04-22T05:55:22.680694+0000","last_clean":"2024-04-22T05:55:22.680694+0000","last_became_active":"2024-04-22T05:50:41.051259+0000","last_became_peered":"2024-04-22T05:50:41.051259+0000","last_unstale":"2024-04-22T05:55:22.680694+0000","last_undegraded":"2024-04-22T05:55:22.680694+0000","last_fullsized":"2024-04-22T05:55:22.680694+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:22.680676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:22.680676+0000","last_clean_scrub_stamp":"2024-04-22T05:55:22.680676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:05:34.545220+0000","scrub_duration":0.001356763,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"23'4981","reported_seq":5111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:29.485246+0000","last_change":"2024-04-22T05:55:29.485517+0000","last_active":"2024-04-22T05:55:29.485246+0000","last_peered":"2024-04-22T05:55:29.485246+0000","last_clean":"2024-04-22T05:55:29.485246+0000","last_became_active":"2024-04-22T05:50:47.119414+0000","last_became_peered":"2024-04-22T05:50:47.119414+0000","last_unstale":"2024-04-22T05:55:29.485246+0000","last_undegraded":"2024-04-22T05:55:29.485246+0000","last_fullsized":"2024-04-22T05:55:29.485246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4981","last_scrub_stamp":"2024-04-22T05:55:29.485120+0000","last_deep_scrub":"23'4981","last_deep_scrub_stamp":"2024-04-22T05:55:29.485120+0000","last_clean_scrub_stamp":"2024-04-22T05:55:29.485120+0000","objects_scrubbed":31,"log_size":4981,"log_dups_size":0,"ondisk_log_size":4981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:41:37.190926+0000","scrub_duration":0.48502272699999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124520800,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":4927,"num_write_kb":205619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:29.064418+0000","last_change":"2024-04-22T05:55:29.064619+0000","last_active":"2024-04-22T05:55:29.064418+0000","last_peered":"2024-04-22T05:55:29.064418+0000","last_clean":"2024-04-22T05:55:29.064418+0000","last_became_active":"2024-04-22T05:50:41.044865+0000","last_became_peered":"2024-04-22T05:50:41.044865+0000","last_unstale":"2024-04-22T05:55:29.064418+0000","last_undegraded":"2024-04-22T05:55:29.064418+0000","last_fullsized":"2024-04-22T05:55:29.064418+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:29.064368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:29.064368+0000","last_clean_scrub_stamp":"2024-04-22T05:55:29.064368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:20:39.181665+0000","scrub_duration":0.003329364,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"23'5463","reported_seq":5492,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:29.818270+0000","last_change":"2024-04-22T05:55:29.818555+0000","last_active":"2024-04-22T05:55:29.818270+0000","last_peered":"2024-04-22T05:55:29.818270+0000","last_clean":"2024-04-22T05:55:29.818270+0000","last_became_active":"2024-04-22T05:50:47.113405+0000","last_became_peered":"2024-04-22T05:50:47.113405+0000","last_unstale":"2024-04-22T05:55:29.818270+0000","last_undegraded":"2024-04-22T05:55:29.818270+0000","last_fullsized":"2024-04-22T05:55:29.818270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5463","last_scrub_stamp":"2024-04-22T05:55:29.818155+0000","last_deep_scrub":"23'5463","last_deep_scrub_stamp":"2024-04-22T05:55:29.818155+0000","last_clean_scrub_stamp":"2024-04-22T05:55:29.818155+0000","objects_scrubbed":32,"log_size":5463,"log_dups_size":0,"ondisk_log_size":5463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:24:10.903144+0000","scrub_duration":0.52685615100000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128647688,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5434,"num_write_kb":203340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:31.546917+0000","last_change":"2024-04-22T05:55:31.547161+0000","last_active":"2024-04-22T05:55:31.546917+0000","last_peered":"2024-04-22T05:55:31.546917+0000","last_clean":"2024-04-22T05:55:31.546917+0000","last_became_active":"2024-04-22T05:50:41.052349+0000","last_became_peered":"2024-04-22T05:50:41.052349+0000","last_unstale":"2024-04-22T05:55:31.546917+0000","last_undegraded":"2024-04-22T05:55:31.546917+0000","last_fullsized":"2024-04-22T05:55:31.546917+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:31.546866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:31.546866+0000","last_clean_scrub_stamp":"2024-04-22T05:55:31.546866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:06:34.832231+0000","scrub_duration":0.0034443590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"23'119","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:28.094886+0000","last_change":"2024-04-22T05:55:28.095221+0000","last_active":"2024-04-22T05:55:28.094886+0000","last_peered":"2024-04-22T05:55:28.094886+0000","last_clean":"2024-04-22T05:55:28.094886+0000","last_became_active":"2024-04-22T05:50:40.046329+0000","last_became_peered":"2024-04-22T05:50:40.046329+0000","last_unstale":"2024-04-22T05:55:28.094886+0000","last_undegraded":"2024-04-22T05:55:28.094886+0000","last_fullsized":"2024-04-22T05:55:28.094886+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'119","last_scrub_stamp":"2024-04-22T05:55:28.094771+0000","last_deep_scrub":"23'119","last_deep_scrub_stamp":"2024-04-22T05:55:28.094771+0000","last_clean_scrub_stamp":"2024-04-22T05:55:28.094771+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:17:52.246384+0000","scrub_duration":0.011123787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"23'3960","reported_seq":3991,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:23.486878+0000","last_change":"2024-04-22T05:55:23.486951+0000","last_active":"2024-04-22T05:55:23.486878+0000","last_peered":"2024-04-22T05:55:23.486878+0000","last_clean":"2024-04-22T05:55:23.486878+0000","last_became_active":"2024-04-22T05:50:47.115449+0000","last_became_peered":"2024-04-22T05:50:47.115449+0000","last_unstale":"2024-04-22T05:55:23.486878+0000","last_undegraded":"2024-04-22T05:55:23.486878+0000","last_fullsized":"2024-04-22T05:55:23.486878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3960","last_scrub_stamp":"2024-04-22T05:55:23.486858+0000","last_deep_scrub":"23'3960","last_deep_scrub_stamp":"2024-04-22T05:55:23.486858+0000","last_clean_scrub_stamp":"2024-04-22T05:55:23.486858+0000","objects_scrubbed":28,"log_size":3960,"log_dups_size":0,"ondisk_log_size":3960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:40.380402+0000","scrub_duration":0.25562303400000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":3867,"num_write_kb":173187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"23'10268","reported_seq":10411,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:27.499815+0000","last_change":"2024-04-22T05:55:27.499971+0000","last_active":"2024-04-22T05:55:27.499815+0000","last_peered":"2024-04-22T05:55:27.499815+0000","last_clean":"2024-04-22T05:55:27.499815+0000","last_became_active":"2024-04-22T05:50:47.173440+0000","last_became_peered":"2024-04-22T05:50:47.173440+0000","last_unstale":"2024-04-22T05:55:27.499815+0000","last_undegraded":"2024-04-22T05:55:27.499815+0000","last_fullsized":"2024-04-22T05:55:27.499815+0000","mapping_epoch":19,"log_start":"21'200","ondisk_log_start":"21'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10268","last_scrub_stamp":"2024-04-22T05:55:27.499781+0000","last_deep_scrub":"23'10268","last_deep_scrub_stamp":"2024-04-22T05:55:27.499781+0000","last_clean_scrub_stamp":"2024-04-22T05:55:27.499781+0000","objects_scrubbed":44,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:53:27.639500+0000","scrub_duration":0.76434783799999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":381,"num_read_kb":346,"num_write":8425,"num_write_kb":237093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"23'5065","reported_seq":5093,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:30.343397+0000","last_change":"2024-04-22T05:55:30.343707+0000","last_active":"2024-04-22T05:55:30.343397+0000","last_peered":"2024-04-22T05:55:30.343397+0000","last_clean":"2024-04-22T05:55:30.343397+0000","last_became_active":"2024-04-22T05:50:47.120541+0000","last_became_peered":"2024-04-22T05:50:47.120541+0000","last_unstale":"2024-04-22T05:55:30.343397+0000","last_undegraded":"2024-04-22T05:55:30.343397+0000","last_fullsized":"2024-04-22T05:55:30.343397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5065","last_scrub_stamp":"2024-04-22T05:55:30.343343+0000","last_deep_scrub":"23'5065","last_deep_scrub_stamp":"2024-04-22T05:55:30.343343+0000","last_clean_scrub_stamp":"2024-04-22T05:55:30.343343+0000","objects_scrubbed":29,"log_size":5065,"log_dups_size":0,"ondisk_log_size":5065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:46:22.673675+0000","scrub_duration":0.51344110700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5009,"num_write_kb":193539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"23'4738","reported_seq":4767,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:30.682420+0000","last_change":"2024-04-22T05:55:30.682612+0000","last_active":"2024-04-22T05:55:30.682420+0000","last_peered":"2024-04-22T05:55:30.682420+0000","last_clean":"2024-04-22T05:55:30.682420+0000","last_became_active":"2024-04-22T05:50:47.123115+0000","last_became_peered":"2024-04-22T05:50:47.123115+0000","last_unstale":"2024-04-22T05:55:30.682420+0000","last_undegraded":"2024-04-22T05:55:30.682420+0000","last_fullsized":"2024-04-22T05:55:30.682420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4738","last_scrub_stamp":"2024-04-22T05:55:30.682374+0000","last_deep_scrub":"23'4738","last_deep_scrub_stamp":"2024-04-22T05:55:30.682374+0000","last_clean_scrub_stamp":"2024-04-22T05:55:30.682374+0000","objects_scrubbed":24,"log_size":4738,"log_dups_size":0,"ondisk_log_size":4738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:37:19.251975+0000","scrub_duration":0.36940600099999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600592,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":4720,"num_write_kb":218508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"23'4402","reported_seq":4432,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:32.933610+0000","last_change":"2024-04-22T05:55:32.933949+0000","last_active":"2024-04-22T05:55:32.933610+0000","last_peered":"2024-04-22T05:55:32.933610+0000","last_clean":"2024-04-22T05:55:32.933610+0000","last_became_active":"2024-04-22T05:50:47.114834+0000","last_became_peered":"2024-04-22T05:50:47.114834+0000","last_unstale":"2024-04-22T05:55:32.933610+0000","last_undegraded":"2024-04-22T05:55:32.933610+0000","last_fullsized":"2024-04-22T05:55:32.933610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4402","last_scrub_stamp":"2024-04-22T05:55:32.933553+0000","last_deep_scrub":"23'4402","last_deep_scrub_stamp":"2024-04-22T05:55:32.933553+0000","last_clean_scrub_stamp":"2024-04-22T05:55:32.933553+0000","objects_scrubbed":24,"log_size":4402,"log_dups_size":0,"ondisk_log_size":4402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:23:14.667949+0000","scrub_duration":0.35749684700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":4338,"num_write_kb":218182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'3579","reported_seq":3607,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:41.289292+0000","last_change":"2024-04-22T05:55:41.289526+0000","last_active":"2024-04-22T05:55:41.289292+0000","last_peered":"2024-04-22T05:55:41.289292+0000","last_clean":"2024-04-22T05:55:41.289292+0000","last_became_active":"2024-04-22T05:50:47.113260+0000","last_became_peered":"2024-04-22T05:50:47.113260+0000","last_unstale":"2024-04-22T05:55:41.289292+0000","last_undegraded":"2024-04-22T05:55:41.289292+0000","last_fullsized":"2024-04-22T05:55:41.289292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3579","last_scrub_stamp":"2024-04-22T05:55:41.289242+0000","last_deep_scrub":"23'3579","last_deep_scrub_stamp":"2024-04-22T05:55:41.289242+0000","last_clean_scrub_stamp":"2024-04-22T05:55:41.289242+0000","objects_scrubbed":25,"log_size":3579,"log_dups_size":0,"ondisk_log_size":3579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:50:00.346236+0000","scrub_duration":0.38064881099999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104202240,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3525,"num_write_kb":160132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'3652","reported_seq":3770,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:42.247106+0000","last_change":"2024-04-22T05:55:42.247333+0000","last_active":"2024-04-22T05:55:42.247106+0000","last_peered":"2024-04-22T05:55:42.247106+0000","last_clean":"2024-04-22T05:55:42.247106+0000","last_became_active":"2024-04-22T05:50:47.122822+0000","last_became_peered":"2024-04-22T05:50:47.122822+0000","last_unstale":"2024-04-22T05:55:42.247106+0000","last_undegraded":"2024-04-22T05:55:42.247106+0000","last_fullsized":"2024-04-22T05:55:42.247106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3652","last_scrub_stamp":"2024-04-22T05:55:42.247060+0000","last_deep_scrub":"23'3652","last_deep_scrub_stamp":"2024-04-22T05:55:42.247060+0000","last_clean_scrub_stamp":"2024-04-22T05:55:42.247060+0000","objects_scrubbed":20,"log_size":3652,"log_dups_size":0,"ondisk_log_size":3652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:17:57.962138+0000","scrub_duration":0.33332855900000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":300,"num_read_kb":235,"num_write":3883,"num_write_kb":108782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'4331","reported_seq":4370,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:18.196701+0000","last_change":"2024-04-22T05:50:47.118515+0000","last_active":"2024-04-22T05:55:18.196701+0000","last_peered":"2024-04-22T05:55:18.196701+0000","last_clean":"2024-04-22T05:55:18.196701+0000","last_became_active":"2024-04-22T05:50:47.117734+0000","last_became_peered":"2024-04-22T05:50:47.117734+0000","last_unstale":"2024-04-22T05:55:18.196701+0000","last_undegraded":"2024-04-22T05:55:18.196701+0000","last_fullsized":"2024-04-22T05:55:18.196701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126091776,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":92,"num_read_kb":134,"num_write":4330,"num_write_kb":222405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"23'3218","reported_seq":3244,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:37.140959+0000","last_change":"2024-04-22T05:55:37.141270+0000","last_active":"2024-04-22T05:55:37.140959+0000","last_peered":"2024-04-22T05:55:37.140959+0000","last_clean":"2024-04-22T05:55:37.140959+0000","last_became_active":"2024-04-22T05:50:47.124711+0000","last_became_peered":"2024-04-22T05:50:47.124711+0000","last_unstale":"2024-04-22T05:55:37.140959+0000","last_undegraded":"2024-04-22T05:55:37.140959+0000","last_fullsized":"2024-04-22T05:55:37.140959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3218","last_scrub_stamp":"2024-04-22T05:55:37.140851+0000","last_deep_scrub":"23'3218","last_deep_scrub_stamp":"2024-04-22T05:55:37.140851+0000","last_clean_scrub_stamp":"2024-04-22T05:55:37.140851+0000","objects_scrubbed":21,"log_size":3218,"log_dups_size":0,"ondisk_log_size":3218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:07:41.912754+0000","scrub_duration":0.27221202,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3149,"num_write_kb":120072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"21'3697","reported_seq":4018,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:24.893735+0000","last_change":"2024-04-22T05:50:47.116263+0000","last_active":"2024-04-22T05:55:24.893735+0000","last_peered":"2024-04-22T05:55:24.893735+0000","last_clean":"2024-04-22T05:55:24.893735+0000","last_became_active":"2024-04-22T05:50:47.115535+0000","last_became_peered":"2024-04-22T05:50:47.115535+0000","last_unstale":"2024-04-22T05:55:24.893735+0000","last_undegraded":"2024-04-22T05:55:24.893735+0000","last_fullsized":"2024-04-22T05:55:24.893735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92406272,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1026,"num_read_kb":919,"num_write":3912,"num_write_kb":173447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"23'3456","reported_seq":3485,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:23.312857+0000","last_change":"2024-04-22T05:55:23.312857+0000","last_active":"2024-04-22T05:55:23.312857+0000","last_peered":"2024-04-22T05:55:23.312857+0000","last_clean":"2024-04-22T05:55:23.312857+0000","last_became_active":"2024-04-22T05:50:47.129345+0000","last_became_peered":"2024-04-22T05:50:47.129345+0000","last_unstale":"2024-04-22T05:55:23.312857+0000","last_undegraded":"2024-04-22T05:55:23.312857+0000","last_fullsized":"2024-04-22T05:55:23.312857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3456","last_scrub_stamp":"2024-04-22T05:55:23.312622+0000","last_deep_scrub":"23'3456","last_deep_scrub_stamp":"2024-04-22T05:55:23.312622+0000","last_clean_scrub_stamp":"2024-04-22T05:55:23.312622+0000","objects_scrubbed":18,"log_size":3456,"log_dups_size":0,"ondisk_log_size":3456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:34:38.998900+0000","scrub_duration":0.21609183200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":71,"num_write":3499,"num_write_kb":148421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"23'6849","reported_seq":6865,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:24.893766+0000","last_change":"2024-04-22T05:50:47.127371+0000","last_active":"2024-04-22T05:55:24.893766+0000","last_peered":"2024-04-22T05:55:24.893766+0000","last_clean":"2024-04-22T05:55:24.893766+0000","last_became_active":"2024-04-22T05:50:47.126563+0000","last_became_peered":"2024-04-22T05:50:47.126563+0000","last_unstale":"2024-04-22T05:55:24.893766+0000","last_undegraded":"2024-04-22T05:55:24.893766+0000","last_fullsized":"2024-04-22T05:55:24.893766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:50:46.073708+0000","last_clean_scrub_stamp":"2024-04-22T05:50:46.073708+0000","objects_scrubbed":0,"log_size":6849,"log_dups_size":0,"ondisk_log_size":6849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1,"num_read_kb":64,"num_write":6824,"num_write_kb":157279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3567505656,"num_objects":877,"num_object_clones":0,"num_object_copies":1754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":877,"num_whiteouts":0,"num_read":2560,"num_read_kb":2576,"num_write":157144,"num_write_kb":5626241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6796083200,"data_stored":6833661048,"data_compressed":194275682,"data_compressed_allocated":202047488,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":159244,"ondisk_log_size":159244,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11158,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":14,"seq":60129542206,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591960,"kb_used_data":408272,"kb_used_omap":1,"kb_used_meta":183678,"kb_avail":93779880,"statfs":{"total":96636764160,"available":96030597120,"internally_reserved":0,"allocated":418070528,"data_stored":419212919,"data_compressed":12754442,"data_compressed_allocated":13279232,"data_compressed_original":14893056,"omap_allocated":1594,"internal_metadata":188086726},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.84599999999999997,"15min":0.78800000000000003},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.95799999999999996,"5min":20.890000000000001,"15min":20.890000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.622,"15min":0.66000000000000003},"min":{"1min":0.22600000000000001,"5min":0.185,"15min":0.185},"max":{"1min":1.032,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.70699999999999996}]},{"osd":1,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66400000000000003,"15min":0.67200000000000004},"min":{"1min":0.24199999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.221,"15min":0.221},"max":{"1min":1.04,"5min":1.218,"15min":1.218},"last":0.78600000000000003}]},{"osd":3,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.747,"15min":0.86499999999999999},"min":{"1min":0.26100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.92700000000000005,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.82099999999999995,"15min":0.88700000000000001},"min":{"1min":0.33000000000000002,"5min":0.246,"15min":0.246},"max":{"1min":1.111,"5min":3.27,"15min":3.27},"last":0.59899999999999998}]},{"osd":4,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81599999999999995,"15min":0.80200000000000005},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.86299999999999999,"15min":0.85499999999999998},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.27,"5min":1.54,"15min":1.54},"last":1.109}]},{"osd":5,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82999999999999996,"15min":0.84599999999999997},"min":{"1min":0.39100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.389,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.36699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.6439999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.081}]},{"osd":6,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.92800000000000005,"15min":0.95399999999999996},"min":{"1min":0.41699999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.204,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":1.155},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94499999999999995,"15min":0.97399999999999998},"min":{"1min":0.39400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.597,"5min":1.651,"15min":1.651},"last":0.93100000000000005}]},{"osd":7,"last update":"Mon Apr 22 05:54:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.2869999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.879,"5min":1.0920000000000001,"15min":1.02},"min":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4750000000000001,"5min":17.995999999999999,"15min":17.995999999999999},"last":0.90100000000000002}]}]},{"osd":6,"up_from":14,"seq":60129542210,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418588,"kb_used_data":311188,"kb_used_omap":1,"kb_used_meta":107390,"kb_avail":93953252,"statfs":{"total":96636764160,"available":96208130048,"internally_reserved":0,"allocated":318656512,"data_stored":318798411,"data_compressed":4811634,"data_compressed_allocated":5001216,"data_compressed_original":5484544,"omap_allocated":1594,"internal_metadata":109967814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":1.506,"15min":1.8380000000000001},"min":{"1min":0.373,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.0720000000000001,"5min":34.100000000000001,"15min":34.100000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":1.0589999999999999,"15min":0.85199999999999998},"min":{"1min":0.439,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.131,"5min":30.306000000000001,"15min":30.306000000000001},"last":0.88700000000000001}]},{"osd":1,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.79200000000000004,"15min":0.73799999999999999},"min":{"1min":0.47799999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.2849999999999999,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.81599999999999995,"15min":0.77600000000000002},"min":{"1min":0.32600000000000001,"5min":0.249,"15min":0.249},"max":{"1min":1.2430000000000001,"5min":1.74,"15min":1.74},"last":1.129}]},{"osd":2,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.88500000000000001,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.216,"15min":0.216},"max":{"1min":1.3009999999999999,"5min":2.488,"15min":2.488},"last":1.169},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.92700000000000005,"15min":0.88700000000000001},"min":{"1min":0.503,"5min":0.31,"15min":0.31},"max":{"1min":1.2370000000000001,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":0.67500000000000004}]},{"osd":3,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84799999999999998,"15min":0.86099999999999999},"min":{"1min":0.56499999999999995,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.101,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0209999999999999,"15min":0.95999999999999996},"min":{"1min":0.45200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.149,"5min":13.539,"15min":13.539},"last":1.0029999999999999}]},{"osd":4,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.89600000000000002,"15min":0.92900000000000005},"min":{"1min":0.60199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.242,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88500000000000001,"15min":0.86699999999999999},"min":{"1min":0.501,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.204,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.048}]},{"osd":5,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.101,"15min":1.2310000000000001},"min":{"1min":0.66700000000000004,"5min":0.312,"15min":0.312},"max":{"1min":1.1970000000000001,"5min":11.922000000000001,"15min":11.922000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1,"15min":0.995},"min":{"1min":0.46999999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3080000000000001,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":1.083}]},{"osd":7,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0269999999999999,"15min":0.995},"min":{"1min":0.60599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.353,"5min":2.129,"15min":2.129},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.101,"15min":1.101},"min":{"1min":0.67300000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3879999999999999,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.96499999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542208,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1090860,"kb_used_data":843236,"kb_used_omap":1,"kb_used_meta":247614,"kb_avail":93280980,"statfs":{"total":96636764160,"available":95519723520,"internally_reserved":0,"allocated":863473664,"data_stored":869120975,"data_compressed":21037680,"data_compressed_allocated":21954560,"data_compressed_original":28217344,"omap_allocated":1594,"internal_metadata":253557190},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76400000000000001,"15min":0.68999999999999995},"min":{"1min":0.443,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.3919999999999999,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.875,"15min":0.91000000000000003},"min":{"1min":0.29299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.3300000000000001,"5min":4.6390000000000002,"15min":4.6390000000000002},"last":1.0980000000000001}]},{"osd":1,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.78900000000000003,"15min":0.71399999999999997},"min":{"1min":0.35799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":1.3500000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":1.0049999999999999,"15min":1.181},"min":{"1min":0.376,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.0449999999999999,"5min":8.202,"15min":8.202},"last":0.80500000000000005}]},{"osd":2,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.83299999999999996,"15min":0.78800000000000003},"min":{"1min":0.39900000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4299999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.90300000000000002,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.474,"5min":1.673,"15min":1.673},"last":1.1519999999999999}]},{"osd":3,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":2.0110000000000001,"15min":2.8210000000000002},"min":{"1min":0.33600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.423,"5min":44.186999999999998,"15min":44.186999999999998},"last":1.3109999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.081,"15min":1.198},"min":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1419999999999999,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.70999999999999996}]},{"osd":4,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.1659999999999999,"15min":1.3240000000000001},"min":{"1min":0.29899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.3520000000000001,"5min":11.438000000000001,"15min":11.438000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.98599999999999999,"15min":0.95999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.171,"5min":2.181,"15min":2.181},"last":0.77300000000000002}]},{"osd":6,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":0.93500000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1830000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.85099999999999998,"5min":1.097,"15min":1.335},"min":{"1min":0.36499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.222,"5min":11.474,"15min":11.474},"last":0.89100000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:54:49 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.107,"15min":1.081},"min":{"1min":0.38200000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.542,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.373},{"interface":"front","average":{"1min":1.01,"5min":1.1100000000000001,"15min":1.1579999999999999},"min":{"1min":0.51700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3009999999999999,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.0549999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1920752,"kb_used_data":1416424,"kb_used_omap":1,"kb_used_meta":504318,"kb_avail":92451088,"statfs":{"total":96636764160,"available":94669914112,"internally_reserved":0,"allocated":1450418176,"data_stored":1463127323,"data_compressed":51243088,"data_compressed_allocated":53432320,"data_compressed_original":67096576,"omap_allocated":1594,"internal_metadata":516422086},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":0.91800000000000004,"15min":0.75800000000000001},"min":{"1min":0.28499999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":12.618,"5min":12.618,"15min":12.618},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.0409999999999999,"15min":0.92000000000000004},"min":{"1min":0.35699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":12.548,"5min":12.548,"15min":12.548},"last":1.052}]},{"osd":1,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.751,"15min":0.755},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.1970000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.5509999999999999,"5min":0.96099999999999997,"15min":0.81799999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":12.449999999999999,"5min":12.449999999999999,"15min":12.449999999999999},"last":0.999}]},{"osd":2,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.33900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.097,"5min":2.226,"15min":2.226},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.048,"15min":0.92800000000000005},"min":{"1min":0.42899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":12.673,"5min":12.673,"15min":12.673},"last":0.92300000000000004}]},{"osd":3,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.90400000000000003,"15min":0.88400000000000001},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.2070000000000001,"5min":3.5329999999999999,"15min":3.5329999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.41899999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.335,"5min":2.254,"15min":2.254},"last":1.248}]},{"osd":5,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.1040000000000001,"15min":1.0349999999999999},"min":{"1min":0.40200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":12.721,"5min":12.721,"15min":12.721},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.95999999999999996},"min":{"1min":0.39500000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.369,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.872}]},{"osd":6,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.1919999999999999,"15min":1.04},"min":{"1min":0.55600000000000005,"5min":0.312,"15min":0.312},"max":{"1min":12.765000000000001,"5min":12.765000000000001,"15min":12.765000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0860000000000001,"15min":1.087},"min":{"1min":0.57699999999999996,"5min":0.372,"15min":0.372},"max":{"1min":1.431,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.1990000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:54:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.038,"15min":1.0569999999999999},"min":{"1min":0.51800000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.456,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.091,"15min":1.0629999999999999},"min":{"1min":0.505,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.6120000000000001,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.175}]}]},{"osd":0,"up_from":14,"seq":60129542210,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1378076,"kb_used_data":1018004,"kb_used_omap":1,"kb_used_meta":360062,"kb_avail":92993764,"statfs":{"total":96636764160,"available":95225614336,"internally_reserved":0,"allocated":1042436096,"data_stored":1047505319,"data_compressed":29839922,"data_compressed_allocated":30982144,"data_compressed_original":36835328,"omap_allocated":1593,"internal_metadata":368703943},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60599999999999998,"15min":0.55000000000000004},"min":{"1min":0.26200000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.792,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.74199999999999999,"15min":0.80600000000000005},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1.4770000000000001,"5min":4.7489999999999997,"15min":4.7489999999999997},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.68700000000000006,"15min":0.625},"min":{"1min":0.28399999999999997,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.90100000000000002},"min":{"1min":0.33200000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.6579999999999999,"5min":4.7880000000000003,"15min":4.7880000000000003},"last":0.70599999999999996}]},{"osd":3,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.5429999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.79400000000000004,"15min":0.76300000000000001},"min":{"1min":0.315,"5min":0.307,"15min":0.307},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.81200000000000006}]},{"osd":4,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.01,"15min":1.085},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.232,"5min":4.8440000000000003,"15min":4.8440000000000003},"last":1.107},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.91400000000000003,"15min":0.90400000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.258,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.752}]},{"osd":5,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.94799999999999995,"15min":0.93200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.47,"5min":2.722,"15min":2.722},"last":1.012},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.0289999999999999,"15min":1.107},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.474,"5min":4.8860000000000001,"15min":4.8860000000000001},"last":0.86899999999999999}]},{"osd":6,"last update":"Mon Apr 22 05:54:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.96199999999999997,"15min":0.88900000000000001},"min":{"1min":0.40000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.5449999999999999,"5min":1.76,"15min":1.76},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.94199999999999995,"15min":0.89300000000000002},"min":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3560000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.3}]},{"osd":7,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2629999999999999,"15min":1.4019999999999999},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.4980000000000002,"5min":8.6839999999999993,"15min":8.6839999999999993},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.1970000000000001,"15min":1.2509999999999999},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.4649999999999999,"5min":4.4649999999999999,"15min":4.4649999999999999},"last":1.3540000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1329560,"kb_used_data":959696,"kb_used_omap":1,"kb_used_meta":369854,"kb_avail":93042280,"statfs":{"total":96636764160,"available":95275294720,"internally_reserved":0,"allocated":982728704,"data_stored":987914375,"data_compressed":23928438,"data_compressed_allocated":24997888,"data_compressed_original":30998528,"omap_allocated":1589,"internal_metadata":378730955},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":1.3640000000000001,"15min":1.0680000000000001},"min":{"1min":0.45700000000000002,"5min":0.224,"15min":0.224},"max":{"1min":1.1839999999999999,"5min":46.598999999999997,"15min":46.598999999999997},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":1.0309999999999999,"15min":1.2709999999999999},"min":{"1min":0.32700000000000001,"5min":0.216,"15min":0.216},"max":{"1min":1.141,"5min":15.144,"15min":15.144},"last":0.70399999999999996}]},{"osd":1,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.80900000000000005,"15min":0.77500000000000002},"min":{"1min":0.35699999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.347,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.018,"5min":0.89700000000000002,"15min":0.82699999999999996},"min":{"1min":0.48599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3300000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.91800000000000004}]},{"osd":2,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.89500000000000002,"15min":0.871},"min":{"1min":0.33000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.262,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.94899999999999995,"15min":0.873},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4419999999999999,"5min":3.387,"15min":3.387},"last":1.125}]},{"osd":3,"last update":"Mon Apr 22 05:54:50 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.999,"15min":0.85899999999999999},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3520000000000001,"5min":7.1699999999999999,"15min":7.1699999999999999},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.89600000000000002,"15min":0.82799999999999996},"min":{"1min":0.45600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.399,"5min":3.6299999999999999,"15min":3.6299999999999999},"last":1.0860000000000001}]},{"osd":4,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.258,"15min":1.2909999999999999},"min":{"1min":0.34399999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.4650000000000001,"5min":9.6620000000000008,"15min":9.6620000000000008},"last":1.423},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.94799999999999995,"15min":0.90200000000000002},"min":{"1min":0.40600000000000003,"5min":0.252,"15min":0.252},"max":{"1min":1.335,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":0.878}]},{"osd":5,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.95399999999999996,"15min":0.89900000000000002},"min":{"1min":0.29299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.54,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1659999999999999,"15min":0.997},"min":{"1min":0.39100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.72,"5min":7.2430000000000003,"15min":7.2430000000000003},"last":1.351}]},{"osd":6,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.04,"15min":0.95599999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5649999999999999,"5min":3.5939999999999999,"15min":3.5939999999999999},"last":1.498},{"interface":"front","average":{"1min":1.208,"5min":1.2450000000000001,"15min":1.0649999999999999},"min":{"1min":0.53400000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.794,"5min":7.2960000000000003,"15min":7.2960000000000003},"last":1.2}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1412656,"kb_used_data":1016552,"kb_used_omap":1,"kb_used_meta":396094,"kb_avail":92959184,"statfs":{"total":96636764160,"available":95190204416,"internally_reserved":0,"allocated":1040949248,"data_stored":1044391251,"data_compressed":24230251,"data_compressed_allocated":25128960,"data_compressed_original":29396992,"omap_allocated":1592,"internal_metadata":405600712},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.69199999999999995,"15min":0.82399999999999995},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":1.079,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":1.4970000000000001,"15min":2.2440000000000002},"min":{"1min":0.28799999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.129,"5min":22.721,"15min":22.721},"last":0.93100000000000005}]},{"osd":1,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.80700000000000005,"15min":0.95099999999999996},"min":{"1min":0.23999999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.5840000000000001,"5min":9.6560000000000006,"15min":9.6560000000000006},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.3100000000000001,"5min":2.028,"15min":2.028},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":1.3009999999999999,"15min":1.619},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.3029999999999999,"5min":22.763999999999999,"15min":22.763999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.94299999999999995,"15min":1.0920000000000001},"min":{"1min":0.313,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.6190000000000002,"5min":9.7739999999999991,"15min":9.7739999999999991},"last":0.76500000000000001}]},{"osd":4,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.7270000000000001,"15min":1.8879999999999999},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.657,"5min":27.384,"15min":27.384},"last":1.034},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.571,"15min":1.931},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.508,"5min":22.824000000000002,"15min":22.824000000000002},"last":1.2729999999999999}]},{"osd":5,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.762,"15min":2.3220000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.4039999999999999,"5min":25.565999999999999,"15min":25.565999999999999},"last":1.21},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.98799999999999999,"15min":0.93999999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.6829999999999998,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.96299999999999997}]},{"osd":6,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.6419999999999999,"15min":1.9570000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.706,"5min":22.794,"15min":22.794},"last":1.115},{"interface":"front","average":{"1min":1.006,"5min":1.171,"15min":1.323},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6240000000000001,"5min":9.9559999999999995,"15min":9.9559999999999995},"last":1.0960000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:54:43 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":2.4089999999999998,"15min":3.1589999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.796,"5min":36.770000000000003,"15min":36.770000000000003},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.4039999999999999,"15min":1.4059999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.9100000000000001,"5min":16.239000000000001,"15min":16.239000000000001},"last":1.5069999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":993392,"kb_used_data":665384,"kb_used_omap":1,"kb_used_meta":327998,"kb_avail":93378448,"statfs":{"total":96636764160,"available":95619530752,"internally_reserved":0,"allocated":681353216,"data_stored":685339487,"data_compressed":26441385,"data_compressed_allocated":27336704,"data_compressed_original":31993856,"omap_allocated":1594,"internal_metadata":335870406},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.66600000000000004,"15min":0.70699999999999996},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.83099999999999996,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.26200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.1679999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.86699999999999999}]},{"osd":2,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.26100000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.014,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.68600000000000005,"15min":0.61799999999999999},"min":{"1min":0.34599999999999997,"5min":0.22,"15min":0.22},"max":{"1min":1.0900000000000001,"5min":3.871,"15min":3.871},"last":1.1459999999999999}]},{"osd":3,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.29699999999999999,"5min":0.252,"15min":0.252},"max":{"1min":1.1970000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.83499999999999996,"15min":0.85199999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.256,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.68400000000000005}]},{"osd":4,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.0409999999999999,"15min":1.1180000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.242,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.044,"15min":1.044},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":2.0880000000000001,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":0.95299999999999996}]},{"osd":5,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.98299999999999998,"15min":0.94499999999999995},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.2650000000000001,"5min":3.8460000000000001,"15min":3.8460000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.98799999999999999,"15min":0.96999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.325,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.1020000000000001}]},{"osd":6,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97699999999999998,"15min":0.92300000000000004},"min":{"1min":0.38400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3999999999999999,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.177},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.98799999999999999,"15min":0.95799999999999996},"min":{"1min":0.42699999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.0720000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:54:45 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.042,"15min":1.004},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.1800000000000002,"5min":3.8159999999999998,"15min":3.8159999999999998},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.0720000000000001,"15min":1.0369999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.615,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.2230000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1042149376,"data_stored":1046890840,"data_compressed":29834343,"data_compressed_allocated":30949376,"data_compressed_original":36323328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":681115648,"data_stored":685252972,"data_compressed":26441385,"data_compressed_allocated":27336704,"data_compressed_original":31993856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417832960,"data_stored":419126404,"data_compressed":12754442,"data_compressed_allocated":13279232,"data_compressed_original":14893056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1040711680,"data_stored":1044304736,"data_compressed":24230251,"data_compressed_allocated":25128960,"data_compressed_original":29396992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1450180608,"data_stored":1463040808,"data_compressed":51243088,"data_compressed_allocated":53432320,"data_compressed_original":67096576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":863236096,"data_stored":869034460,"data_compressed":21037680,"data_compressed_allocated":21954560,"data_compressed_original":28217344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":318418944,"data_stored":318711896,"data_compressed":4811634,"data_compressed_allocated":5001216,"data_compressed_original":5484544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":982437888,"data_stored":987298932,"data_compressed":23922859,"data_compressed_allocated":24965120,"data_compressed_original":30486528,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T05:55:45.592 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:45.592 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:45.593 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:45.593 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:45.593 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:45.593 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:45.594 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-22T05:50:46.073708+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=50, tm_sec=46, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=5, tm_min=55, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T05:55:45.594 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-22T05:56:05.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T05:56:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc42448640 1 Processor -- start 2024-04-22T05:56:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc42448640 1 -- start start 2024-04-22T05:56:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc42448640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc3c06b230 0x7fdc3c071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:56:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc42448640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc3c13e2e0 0x7fdc3c13e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:56:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc42448640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3c070cc0 con 0x7fdc3c144a80 2024-04-22T05:56:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc42448640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3c06b610 con 0x7fdc3c06b230 2024-04-22T05:56:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc42448640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3c0ac1a0 con 0x7fdc3c13e2e0 2024-04-22T05:56:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc3bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc3c06b230 0x7fdc3c071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:56:05.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc409e5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdc3c144a80 0x7fdc3c144ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42460/0 (socket says 172.21.15.40:42460) 2024-04-22T05:56:05.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc409e5640 1 -- 172.21.15.40:0/2463077029 learned_addr learned my addr 172.21.15.40:0/2463077029 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:56:05.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc33fff640 1 --2- 172.21.15.40:0/2463077029 >> v2:172.21.15.62:3300/0 conn(0x7fdc3c13e2e0 0x7fdc3c13e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:56:05.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc3bfff640 1 -- 172.21.15.40:0/2463077029 >> v2:172.21.15.62:3300/0 conn(0x7fdc3c13e2e0 msgr2=0x7fdc3c13e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:56:05.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc3bfff640 1 --2- 172.21.15.40:0/2463077029 >> v2:172.21.15.62:3300/0 conn(0x7fdc3c13e2e0 0x7fdc3c13e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:05.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc3bfff640 1 -- 172.21.15.40:0/2463077029 >> v1:172.21.15.40:6789/0 conn(0x7fdc3c144a80 legacy=0x7fdc3c144ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:56:05.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc3b7fe640 1 -- 172.21.15.40:0/2463077029 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172963103 0 0) 0x7fdc3c070cc0 con 0x7fdc3c144a80 2024-04-22T05:56:05.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.726+0000 7fdc3bfff640 1 -- 172.21.15.40:0/2463077029 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c0713e0 con 0x7fdc3c06b230 2024-04-22T05:56:05.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc3bfff640 1 --2- 172.21.15.40:0/2463077029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc3c06b230 0x7fdc3c071850 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0b4c50 tx=0x7fdc2c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83df54f5670a40a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:56:05.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc3b7fe640 1 -- 172.21.15.40:0/2463077029 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c0ab5f0 con 0x7fdc3c06b230 2024-04-22T05:56:05.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc3b7fe640 1 -- 172.21.15.40:0/2463077029 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c0ab750 con 0x7fdc3c06b230 2024-04-22T05:56:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc3b7fe640 1 -- 172.21.15.40:0/2463077029 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c0abae0 con 0x7fdc3c06b230 2024-04-22T05:56:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 -- 172.21.15.40:0/2463077029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc3c06b230 msgr2=0x7fdc3c071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:56:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 --2- 172.21.15.40:0/2463077029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc3c06b230 0x7fdc3c071850 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0b4c50 tx=0x7fdc2c01c3d0 comp rx=0 tx=0).stop 2024-04-22T05:56:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 -- 172.21.15.40:0/2463077029 shutdown_connections 2024-04-22T05:56:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 --2- 172.21.15.40:0/2463077029 >> v2:172.21.15.62:3300/0 conn(0x7fdc3c13e2e0 0x7fdc3c13e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 --2- 172.21.15.40:0/2463077029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc3c06b230 0x7fdc3c071850 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 -- 172.21.15.40:0/2463077029 >> 172.21.15.40:0/2463077029 conn(0x7fdc3c0b3fb0 msgr2=0x7fdc3c0b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:56:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 -- 172.21.15.40:0/2463077029 shutdown_connections 2024-04-22T05:56:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 -- 172.21.15.40:0/2463077029 wait complete. 2024-04-22T05:56:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 Processor -- start 2024-04-22T05:56:05.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 -- start start 2024-04-22T05:56:05.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc3c06b230 0x7fdc3c140830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:56:05.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc3c13e2e0 0x7fdc3c13c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:56:05.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc3c144a80 0x7fdc3c12f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:56:05.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c13cb60 con 0x7fdc3c144a80 2024-04-22T05:56:05.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c13d840 con 0x7fdc3c13e2e0 2024-04-22T05:56:05.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c13db20 con 0x7fdc3c06b230 2024-04-22T05:56:05.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc409e5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc3c144a80 0x7fdc3c12f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:56:05.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc3bfff640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc3c06b230 0x7fdc3c140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:56:05.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc409e5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc3c144a80 0x7fdc3c12f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53970/0 (socket says 172.21.15.40:53970) 2024-04-22T05:56:05.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc409e5640 1 -- 172.21.15.40:0/3912531037 learned_addr learned my addr 172.21.15.40:0/3912531037 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:56:05.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc33fff640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc3c13e2e0 0x7fdc3c13c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:56:05.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc409e5640 1 -- 172.21.15.40:0/3912531037 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc3c06b230 msgr2=0x7fdc3c140830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:56:05.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc409e5640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc3c06b230 0x7fdc3c140830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:05.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc409e5640 1 -- 172.21.15.40:0/3912531037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc3c13e2e0 msgr2=0x7fdc3c13c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:56:05.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc409e5640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc3c13e2e0 0x7fdc3c13c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:05.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc409e5640 1 -- 172.21.15.40:0/3912531037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c009420 con 0x7fdc3c144a80 2024-04-22T05:56:05.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc33fff640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc3c13e2e0 0x7fdc3c13c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:56:05.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc409e5640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc3c144a80 0x7fdc3c12f6d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400c700 tx=0x7fdc3400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:56:05.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc397fa640 1 -- 172.21.15.40:0/3912531037 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3400ce10 con 0x7fdc3c144a80 2024-04-22T05:56:05.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc397fa640 1 -- 172.21.15.40:0/3912531037 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3400a640 con 0x7fdc3c144a80 2024-04-22T05:56:05.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc42448640 1 -- 172.21.15.40:0/3912531037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c13cdb0 con 0x7fdc3c144a80 2024-04-22T05:56:05.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.730+0000 7fdc397fa640 1 -- 172.21.15.40:0/3912531037 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3400aa70 con 0x7fdc3c144a80 2024-04-22T05:56:05.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.734+0000 7fdc42448640 1 -- 172.21.15.40:0/3912531037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c134d60 con 0x7fdc3c144a80 2024-04-22T05:56:05.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.734+0000 7fdc42448640 1 -- 172.21.15.40:0/3912531037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3c071eb0 con 0x7fdc3c144a80 2024-04-22T05:56:05.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.734+0000 7fdc397fa640 1 -- 172.21.15.40:0/3912531037 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc3400abd0 con 0x7fdc3c144a80 2024-04-22T05:56:05.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.734+0000 7fdc397fa640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fdc1c066690 0x7fdc1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:56:05.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.734+0000 7fdc397fa640 1 -- 172.21.15.40:0/3912531037 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7fdc34083720 con 0x7fdc3c144a80 2024-04-22T05:56:05.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.734+0000 7fdc3bfff640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fdc1c066690 0x7fdc1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:56:05.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.734+0000 7fdc3bfff640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fdc1c066690 0x7fdc1c068b50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0af000 tx=0x7fdc2c0a7000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:56:05.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.738+0000 7fdc397fa640 1 -- 172.21.15.40:0/3912531037 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7fdc34050c10 con 0x7fdc3c144a80 2024-04-22T05:56:05.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.894+0000 7fdc42448640 1 -- 172.21.15.40:0/3912531037 --> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc3c145810 con 0x7fdc1c066690 2024-04-22T05:56:05.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.910+0000 7fdc397fa640 1 -- 172.21.15.40:0/3912531037 <== mgr.4111 v2:172.21.15.62:6832/24860 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+154517 (secure 0 0 0) 0x7fdc2c0ab5d0 con 0x7fdc1c066690 2024-04-22T05:56:05.911 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-22T05:56:05.912 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-22T05:56:05.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.910+0000 7fdc42448640 1 -- 172.21.15.40:0/3912531037 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fdc1c066690 msgr2=0x7fdc1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:56:05.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.910+0000 7fdc42448640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fdc1c066690 0x7fdc1c068b50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0af000 tx=0x7fdc2c0a7000 comp rx=0 tx=0).stop 2024-04-22T05:56:05.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.910+0000 7fdc42448640 1 -- 172.21.15.40:0/3912531037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc3c144a80 msgr2=0x7fdc3c12f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:56:05.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.910+0000 7fdc42448640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc3c144a80 0x7fdc3c12f6d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400c700 tx=0x7fdc3400cbc0 comp rx=0 tx=0).stop 2024-04-22T05:56:05.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.910+0000 7fdc42448640 1 -- 172.21.15.40:0/3912531037 shutdown_connections 2024-04-22T05:56:05.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.910+0000 7fdc42448640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7fdc1c066690 0x7fdc1c068b50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:05.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.910+0000 7fdc42448640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc3c13e2e0 0x7fdc3c13c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:05.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.910+0000 7fdc42448640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc3c144a80 0x7fdc3c12f6d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:05.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.910+0000 7fdc42448640 1 --2- 172.21.15.40:0/3912531037 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7fdc3c06b230 0x7fdc3c140830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:05.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.910+0000 7fdc42448640 1 -- 172.21.15.40:0/3912531037 >> 172.21.15.40:0/3912531037 conn(0x7fdc3c0b3fb0 msgr2=0x7fdc3c0b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:56:05.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.910+0000 7fdc42448640 1 -- 172.21.15.40:0/3912531037 shutdown_connections 2024-04-22T05:56:05.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:05.910+0000 7fdc42448640 1 -- 172.21.15.40:0/3912531037 wait complete. 2024-04-22T05:56:05.930 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-04-22T05:56:05.040156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3568096024,"num_objects":879,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":879,"num_whiteouts":0,"num_read":2676,"num_read_kb":2676,"num_write":157303,"num_write_kb":5628657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159536,"ondisk_log_size":159536,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9139940,"kb_used_data":6638756,"kb_used_omap":12,"kb_used_meta":2501107,"kb_avail":745834780,"statfs":{"total":773094113280,"available":763734814720,"internally_reserved":0,"allocated":6798086144,"data_stored":6835410060,"data_compressed":194286840,"data_compressed_allocated":202113024,"data_compressed_original":244916224,"omap_allocated":12744,"internal_metadata":2561134136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003842"},"pg_stats":[{"pgid":"3.1f","version":"23'3351","reported_seq":3392,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:37.413059+0000","last_change":"2024-04-22T05:55:37.413304+0000","last_active":"2024-04-22T05:55:37.413059+0000","last_peered":"2024-04-22T05:55:37.413059+0000","last_clean":"2024-04-22T05:55:37.413059+0000","last_became_active":"2024-04-22T05:50:47.120501+0000","last_became_peered":"2024-04-22T05:50:47.120501+0000","last_unstale":"2024-04-22T05:55:37.413059+0000","last_undegraded":"2024-04-22T05:55:37.413059+0000","last_fullsized":"2024-04-22T05:55:37.413059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3351","last_scrub_stamp":"2024-04-22T05:55:37.412994+0000","last_deep_scrub":"23'3351","last_deep_scrub_stamp":"2024-04-22T05:55:37.412994+0000","last_clean_scrub_stamp":"2024-04-22T05:55:37.412994+0000","objects_scrubbed":29,"log_size":3351,"log_dups_size":0,"ondisk_log_size":3351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:04:03.177160+0000","scrub_duration":0.41623528900000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109323296,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":3347,"num_write_kb":168857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'5529","reported_seq":5565,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:35.652551+0000","last_change":"2024-04-22T05:55:35.652789+0000","last_active":"2024-04-22T05:55:35.652551+0000","last_peered":"2024-04-22T05:55:35.652551+0000","last_clean":"2024-04-22T05:55:35.652551+0000","last_became_active":"2024-04-22T05:50:47.117274+0000","last_became_peered":"2024-04-22T05:50:47.117274+0000","last_unstale":"2024-04-22T05:55:35.652551+0000","last_undegraded":"2024-04-22T05:55:35.652551+0000","last_fullsized":"2024-04-22T05:55:35.652551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5529","last_scrub_stamp":"2024-04-22T05:55:35.652500+0000","last_deep_scrub":"23'5529","last_deep_scrub_stamp":"2024-04-22T05:55:35.652500+0000","last_clean_scrub_stamp":"2024-04-22T05:55:35.652500+0000","objects_scrubbed":32,"log_size":5529,"log_dups_size":0,"ondisk_log_size":5529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:24.731938+0000","scrub_duration":0.43440045300000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125960704,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":28,"num_read_kb":84,"num_write":5488,"num_write_kb":185156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"23'3436","reported_seq":3473,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:49.753459+0000","last_change":"2024-04-22T05:55:49.753687+0000","last_active":"2024-04-22T05:55:49.753459+0000","last_peered":"2024-04-22T05:55:49.753459+0000","last_clean":"2024-04-22T05:55:49.753459+0000","last_became_active":"2024-04-22T05:50:47.120601+0000","last_became_peered":"2024-04-22T05:50:47.120601+0000","last_unstale":"2024-04-22T05:55:49.753459+0000","last_undegraded":"2024-04-22T05:55:49.753459+0000","last_fullsized":"2024-04-22T05:55:49.753459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3436","last_scrub_stamp":"2024-04-22T05:55:49.753406+0000","last_deep_scrub":"23'3436","last_deep_scrub_stamp":"2024-04-22T05:55:49.753406+0000","last_clean_scrub_stamp":"2024-04-22T05:55:49.753406+0000","objects_scrubbed":18,"log_size":3436,"log_dups_size":0,"ondisk_log_size":3436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:17:46.957123+0000","scrub_duration":0.28913594100000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72812200,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":42,"num_read_kb":34,"num_write":3406,"num_write_kb":94339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"23'3003","reported_seq":3029,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:52.834489+0000","last_change":"2024-04-22T05:55:52.834816+0000","last_active":"2024-04-22T05:55:52.834489+0000","last_peered":"2024-04-22T05:55:52.834489+0000","last_clean":"2024-04-22T05:55:52.834489+0000","last_became_active":"2024-04-22T05:50:47.115803+0000","last_became_peered":"2024-04-22T05:50:47.115803+0000","last_unstale":"2024-04-22T05:55:52.834489+0000","last_undegraded":"2024-04-22T05:55:52.834489+0000","last_fullsized":"2024-04-22T05:55:52.834489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3003","last_scrub_stamp":"2024-04-22T05:55:52.834429+0000","last_deep_scrub":"23'3003","last_deep_scrub_stamp":"2024-04-22T05:55:52.834429+0000","last_clean_scrub_stamp":"2024-04-22T05:55:52.834429+0000","objects_scrubbed":22,"log_size":3003,"log_dups_size":0,"ondisk_log_size":3003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:36:03.814657+0000","scrub_duration":0.284103993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89522176,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2976,"num_write_kb":149441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"23'5258","reported_seq":5309,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:24.558832+0000","last_change":"2024-04-22T05:55:24.558948+0000","last_active":"2024-04-22T05:55:24.558832+0000","last_peered":"2024-04-22T05:55:24.558832+0000","last_clean":"2024-04-22T05:55:24.558832+0000","last_became_active":"2024-04-22T05:50:47.123073+0000","last_became_peered":"2024-04-22T05:50:47.123073+0000","last_unstale":"2024-04-22T05:55:24.558832+0000","last_undegraded":"2024-04-22T05:55:24.558832+0000","last_fullsized":"2024-04-22T05:55:24.558832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5258","last_scrub_stamp":"2024-04-22T05:55:24.558811+0000","last_deep_scrub":"23'5258","last_deep_scrub_stamp":"2024-04-22T05:55:24.558811+0000","last_clean_scrub_stamp":"2024-04-22T05:55:24.558811+0000","objects_scrubbed":36,"log_size":5258,"log_dups_size":0,"ondisk_log_size":5258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:50:19.675491+0000","scrub_duration":0.50381928200000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150994944,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":90,"num_read_kb":78,"num_write":5209,"num_write_kb":201924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"23'6531","reported_seq":6567,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:30.328890+0000","last_change":"2024-04-22T05:55:30.328890+0000","last_active":"2024-04-22T05:55:30.328890+0000","last_peered":"2024-04-22T05:55:30.328890+0000","last_clean":"2024-04-22T05:55:30.328890+0000","last_became_active":"2024-04-22T05:50:47.118038+0000","last_became_peered":"2024-04-22T05:50:47.118038+0000","last_unstale":"2024-04-22T05:55:30.328890+0000","last_undegraded":"2024-04-22T05:55:30.328890+0000","last_fullsized":"2024-04-22T05:55:30.328890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6531","last_scrub_stamp":"2024-04-22T05:55:30.328593+0000","last_deep_scrub":"23'6531","last_deep_scrub_stamp":"2024-04-22T05:55:30.328593+0000","last_clean_scrub_stamp":"2024-04-22T05:55:30.328593+0000","objects_scrubbed":28,"log_size":6531,"log_dups_size":0,"ondisk_log_size":6531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:22.808632+0000","scrub_duration":0.56155419200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113311744,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":6483,"num_write_kb":139343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"23'4782","reported_seq":4810,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:42.670477+0000","last_change":"2024-04-22T05:55:42.670599+0000","last_active":"2024-04-22T05:55:42.670477+0000","last_peered":"2024-04-22T05:55:42.670477+0000","last_clean":"2024-04-22T05:55:42.670477+0000","last_became_active":"2024-04-22T05:50:47.113368+0000","last_became_peered":"2024-04-22T05:50:47.113368+0000","last_unstale":"2024-04-22T05:55:42.670477+0000","last_undegraded":"2024-04-22T05:55:42.670477+0000","last_fullsized":"2024-04-22T05:55:42.670477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4782","last_scrub_stamp":"2024-04-22T05:55:42.670447+0000","last_deep_scrub":"23'4782","last_deep_scrub_stamp":"2024-04-22T05:55:42.670447+0000","last_clean_scrub_stamp":"2024-04-22T05:55:42.670447+0000","objects_scrubbed":27,"log_size":4782,"log_dups_size":0,"ondisk_log_size":4782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:19:50.199229+0000","scrub_duration":0.46783407300000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112566272,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":123245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"23'8079","reported_seq":8114,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:40.814520+0000","last_change":"2024-04-22T05:55:40.814693+0000","last_active":"2024-04-22T05:55:40.814520+0000","last_peered":"2024-04-22T05:55:40.814520+0000","last_clean":"2024-04-22T05:55:40.814520+0000","last_became_active":"2024-04-22T05:50:47.113569+0000","last_became_peered":"2024-04-22T05:50:47.113569+0000","last_unstale":"2024-04-22T05:55:40.814520+0000","last_undegraded":"2024-04-22T05:55:40.814520+0000","last_fullsized":"2024-04-22T05:55:40.814520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8079","last_scrub_stamp":"2024-04-22T05:55:40.814475+0000","last_deep_scrub":"23'8079","last_deep_scrub_stamp":"2024-04-22T05:55:40.814475+0000","last_clean_scrub_stamp":"2024-04-22T05:55:40.814475+0000","objects_scrubbed":26,"log_size":8079,"log_dups_size":0,"ondisk_log_size":8079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:13:10.549067+0000","scrub_duration":0.57767474900000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":8065,"num_write_kb":185808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"23'6498","reported_seq":6537,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:34.396028+0000","last_change":"2024-04-22T05:55:34.396222+0000","last_active":"2024-04-22T05:55:34.396028+0000","last_peered":"2024-04-22T05:55:34.396028+0000","last_clean":"2024-04-22T05:55:34.396028+0000","last_became_active":"2024-04-22T05:50:47.121332+0000","last_became_peered":"2024-04-22T05:50:47.121332+0000","last_unstale":"2024-04-22T05:55:34.396028+0000","last_undegraded":"2024-04-22T05:55:34.396028+0000","last_fullsized":"2024-04-22T05:55:34.396028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6498","last_scrub_stamp":"2024-04-22T05:55:34.395983+0000","last_deep_scrub":"23'6498","last_deep_scrub_stamp":"2024-04-22T05:55:34.395983+0000","last_clean_scrub_stamp":"2024-04-22T05:55:34.395983+0000","objects_scrubbed":27,"log_size":6498,"log_dups_size":0,"ondisk_log_size":6498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:07.124227+0000","scrub_duration":0.428001927,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111009792,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":6495,"num_write_kb":211703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"23'3787","reported_seq":3817,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:46.796836+0000","last_change":"2024-04-22T05:55:46.797016+0000","last_active":"2024-04-22T05:55:46.796836+0000","last_peered":"2024-04-22T05:55:46.796836+0000","last_clean":"2024-04-22T05:55:46.796836+0000","last_became_active":"2024-04-22T05:50:47.119031+0000","last_became_peered":"2024-04-22T05:50:47.119031+0000","last_unstale":"2024-04-22T05:55:46.796836+0000","last_undegraded":"2024-04-22T05:55:46.796836+0000","last_fullsized":"2024-04-22T05:55:46.796836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3787","last_scrub_stamp":"2024-04-22T05:55:46.796791+0000","last_deep_scrub":"23'3787","last_deep_scrub_stamp":"2024-04-22T05:55:46.796791+0000","last_clean_scrub_stamp":"2024-04-22T05:55:46.796791+0000","objects_scrubbed":25,"log_size":3787,"log_dups_size":0,"ondisk_log_size":3787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:56:33.078105+0000","scrub_duration":0.36839992199999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":3747,"num_write_kb":173061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"23'7989","reported_seq":8017,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:48.341162+0000","last_change":"2024-04-22T05:55:48.341349+0000","last_active":"2024-04-22T05:55:48.341162+0000","last_peered":"2024-04-22T05:55:48.341162+0000","last_clean":"2024-04-22T05:55:48.341162+0000","last_became_active":"2024-04-22T05:50:47.118399+0000","last_became_peered":"2024-04-22T05:50:47.118399+0000","last_unstale":"2024-04-22T05:55:48.341162+0000","last_undegraded":"2024-04-22T05:55:48.341162+0000","last_fullsized":"2024-04-22T05:55:48.341162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7989","last_scrub_stamp":"2024-04-22T05:55:48.341122+0000","last_deep_scrub":"23'7989","last_deep_scrub_stamp":"2024-04-22T05:55:48.341122+0000","last_clean_scrub_stamp":"2024-04-22T05:55:48.341122+0000","objects_scrubbed":27,"log_size":7989,"log_dups_size":0,"ondisk_log_size":7989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:51:47.678024+0000","scrub_duration":0.47681926400000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7958,"num_write_kb":234882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"23'7086","reported_seq":7143,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:28.452359+0000","last_change":"2024-04-22T05:55:28.452359+0000","last_active":"2024-04-22T05:55:28.452359+0000","last_peered":"2024-04-22T05:55:28.452359+0000","last_clean":"2024-04-22T05:55:28.452359+0000","last_became_active":"2024-04-22T05:50:47.130842+0000","last_became_peered":"2024-04-22T05:50:47.130842+0000","last_unstale":"2024-04-22T05:55:28.452359+0000","last_undegraded":"2024-04-22T05:55:28.452359+0000","last_fullsized":"2024-04-22T05:55:28.452359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7086","last_scrub_stamp":"2024-04-22T05:55:28.452110+0000","last_deep_scrub":"23'7086","last_deep_scrub_stamp":"2024-04-22T05:55:28.452110+0000","last_clean_scrub_stamp":"2024-04-22T05:55:28.452110+0000","objects_scrubbed":38,"log_size":7086,"log_dups_size":0,"ondisk_log_size":7086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:22:37.268459+0000","scrub_duration":0.68650581600000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149815296,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":118,"num_read_kb":94,"num_write":7055,"num_write_kb":232332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:28.855161+0000","last_change":"2024-04-22T05:55:28.855464+0000","last_active":"2024-04-22T05:55:28.855161+0000","last_peered":"2024-04-22T05:55:28.855161+0000","last_clean":"2024-04-22T05:55:28.855161+0000","last_became_active":"2024-04-22T05:50:41.049696+0000","last_became_peered":"2024-04-22T05:50:41.049696+0000","last_unstale":"2024-04-22T05:55:28.855161+0000","last_undegraded":"2024-04-22T05:55:28.855161+0000","last_fullsized":"2024-04-22T05:55:28.855161+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:28.855062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:28.855062+0000","last_clean_scrub_stamp":"2024-04-22T05:55:28.855062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:51:54.355467+0000","scrub_duration":0.003271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'4474","reported_seq":4522,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:26.512866+0000","last_change":"2024-04-22T05:55:26.513066+0000","last_active":"2024-04-22T05:55:26.512866+0000","last_peered":"2024-04-22T05:55:26.512866+0000","last_clean":"2024-04-22T05:55:26.512866+0000","last_became_active":"2024-04-22T05:50:47.113838+0000","last_became_peered":"2024-04-22T05:50:47.113838+0000","last_unstale":"2024-04-22T05:55:26.512866+0000","last_undegraded":"2024-04-22T05:55:26.512866+0000","last_fullsized":"2024-04-22T05:55:26.512866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4474","last_scrub_stamp":"2024-04-22T05:55:26.512811+0000","last_deep_scrub":"23'4474","last_deep_scrub_stamp":"2024-04-22T05:55:26.512811+0000","last_clean_scrub_stamp":"2024-04-22T05:55:26.512811+0000","objects_scrubbed":30,"log_size":4474,"log_dups_size":0,"ondisk_log_size":4474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:29:16.427953+0000","scrub_duration":0.50539391899999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120194896,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":4449,"num_write_kb":155490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:31.181271+0000","last_change":"2024-04-22T05:55:31.181457+0000","last_active":"2024-04-22T05:55:31.181271+0000","last_peered":"2024-04-22T05:55:31.181271+0000","last_clean":"2024-04-22T05:55:31.181271+0000","last_became_active":"2024-04-22T05:50:41.048628+0000","last_became_peered":"2024-04-22T05:50:41.048628+0000","last_unstale":"2024-04-22T05:55:31.181271+0000","last_undegraded":"2024-04-22T05:55:31.181271+0000","last_fullsized":"2024-04-22T05:55:31.181271+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:31.181223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:31.181223+0000","last_clean_scrub_stamp":"2024-04-22T05:55:31.181223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:49:42.189479+0000","scrub_duration":0.0030718529999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'3274","reported_seq":3302,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:33.534972+0000","last_change":"2024-04-22T05:55:33.535217+0000","last_active":"2024-04-22T05:55:33.534972+0000","last_peered":"2024-04-22T05:55:33.534972+0000","last_clean":"2024-04-22T05:55:33.534972+0000","last_became_active":"2024-04-22T05:50:47.129322+0000","last_became_peered":"2024-04-22T05:50:47.129322+0000","last_unstale":"2024-04-22T05:55:33.534972+0000","last_undegraded":"2024-04-22T05:55:33.534972+0000","last_fullsized":"2024-04-22T05:55:33.534972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3274","last_scrub_stamp":"2024-04-22T05:55:33.534920+0000","last_deep_scrub":"23'3274","last_deep_scrub_stamp":"2024-04-22T05:55:33.534920+0000","last_clean_scrub_stamp":"2024-04-22T05:55:33.534920+0000","objects_scrubbed":20,"log_size":3274,"log_dups_size":0,"ondisk_log_size":3274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:52.910713+0000","scrub_duration":0.28762188999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":3214,"num_write_kb":155139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:35.946947+0000","last_change":"2024-04-22T05:55:35.947194+0000","last_active":"2024-04-22T05:55:35.946947+0000","last_peered":"2024-04-22T05:55:35.946947+0000","last_clean":"2024-04-22T05:55:35.946947+0000","last_became_active":"2024-04-22T05:50:41.340643+0000","last_became_peered":"2024-04-22T05:50:41.340643+0000","last_unstale":"2024-04-22T05:55:35.946947+0000","last_undegraded":"2024-04-22T05:55:35.946947+0000","last_fullsized":"2024-04-22T05:55:35.946947+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:35.946899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:35.946899+0000","last_clean_scrub_stamp":"2024-04-22T05:55:35.946899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:08:49.875131+0000","scrub_duration":0.0037993990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'6389","reported_seq":6423,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:39.727480+0000","last_change":"2024-04-22T05:55:39.727688+0000","last_active":"2024-04-22T05:55:39.727480+0000","last_peered":"2024-04-22T05:55:39.727480+0000","last_clean":"2024-04-22T05:55:39.727480+0000","last_became_active":"2024-04-22T05:50:47.172121+0000","last_became_peered":"2024-04-22T05:50:47.172121+0000","last_unstale":"2024-04-22T05:55:39.727480+0000","last_undegraded":"2024-04-22T05:55:39.727480+0000","last_fullsized":"2024-04-22T05:55:39.727480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6389","last_scrub_stamp":"2024-04-22T05:55:39.727424+0000","last_deep_scrub":"23'6389","last_deep_scrub_stamp":"2024-04-22T05:55:39.727424+0000","last_clean_scrub_stamp":"2024-04-22T05:55:39.727424+0000","objects_scrubbed":28,"log_size":6389,"log_dups_size":0,"ondisk_log_size":6389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:56:50.890766+0000","scrub_duration":0.53035004900000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":26,"num_read_kb":18,"num_write":6371,"num_write_kb":148611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:23.316432+0000","last_change":"2024-04-22T05:55:23.316534+0000","last_active":"2024-04-22T05:55:23.316432+0000","last_peered":"2024-04-22T05:55:23.316432+0000","last_clean":"2024-04-22T05:55:23.316432+0000","last_became_active":"2024-04-22T05:50:41.341254+0000","last_became_peered":"2024-04-22T05:50:41.341254+0000","last_unstale":"2024-04-22T05:55:23.316432+0000","last_undegraded":"2024-04-22T05:55:23.316432+0000","last_fullsized":"2024-04-22T05:55:23.316432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:23.316401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:23.316401+0000","last_clean_scrub_stamp":"2024-04-22T05:55:23.316401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:08:51.088728+0000","scrub_duration":0.00144433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'3329","reported_seq":3366,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:42.791313+0000","last_change":"2024-04-22T05:55:42.791521+0000","last_active":"2024-04-22T05:55:42.791313+0000","last_peered":"2024-04-22T05:55:42.791313+0000","last_clean":"2024-04-22T05:55:42.791313+0000","last_became_active":"2024-04-22T05:50:47.120072+0000","last_became_peered":"2024-04-22T05:50:47.120072+0000","last_unstale":"2024-04-22T05:55:42.791313+0000","last_undegraded":"2024-04-22T05:55:42.791313+0000","last_fullsized":"2024-04-22T05:55:42.791313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3329","last_scrub_stamp":"2024-04-22T05:55:42.791269+0000","last_deep_scrub":"23'3329","last_deep_scrub_stamp":"2024-04-22T05:55:42.791269+0000","last_clean_scrub_stamp":"2024-04-22T05:55:42.791269+0000","objects_scrubbed":24,"log_size":3329,"log_dups_size":0,"ondisk_log_size":3329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:44:33.469332+0000","scrub_duration":0.32450623299999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99223376,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":3325,"num_write_kb":171289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'3657","reported_seq":3694,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:36.200776+0000","last_change":"2024-04-22T05:55:36.201048+0000","last_active":"2024-04-22T05:55:36.200776+0000","last_peered":"2024-04-22T05:55:36.200776+0000","last_clean":"2024-04-22T05:55:36.200776+0000","last_became_active":"2024-04-22T05:50:47.124841+0000","last_became_peered":"2024-04-22T05:50:47.124841+0000","last_unstale":"2024-04-22T05:55:36.200776+0000","last_undegraded":"2024-04-22T05:55:36.200776+0000","last_fullsized":"2024-04-22T05:55:36.200776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3657","last_scrub_stamp":"2024-04-22T05:55:36.200720+0000","last_deep_scrub":"23'3657","last_deep_scrub_stamp":"2024-04-22T05:55:36.200720+0000","last_clean_scrub_stamp":"2024-04-22T05:55:36.200720+0000","objects_scrubbed":28,"log_size":3657,"log_dups_size":0,"ondisk_log_size":3657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:46.227043+0000","scrub_duration":0.37396192499999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":3649,"num_write_kb":178556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:25.655509+0000","last_change":"2024-04-22T05:55:25.655709+0000","last_active":"2024-04-22T05:55:25.655509+0000","last_peered":"2024-04-22T05:55:25.655509+0000","last_clean":"2024-04-22T05:55:25.655509+0000","last_became_active":"2024-04-22T05:50:41.050711+0000","last_became_peered":"2024-04-22T05:50:41.050711+0000","last_unstale":"2024-04-22T05:55:25.655509+0000","last_undegraded":"2024-04-22T05:55:25.655509+0000","last_fullsized":"2024-04-22T05:55:25.655509+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:25.655461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:25.655461+0000","last_clean_scrub_stamp":"2024-04-22T05:55:25.655461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:14:28.626354+0000","scrub_duration":0.0034954550000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"23'5425","reported_seq":5456,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:37.732937+0000","last_change":"2024-04-22T05:55:37.733151+0000","last_active":"2024-04-22T05:55:37.732937+0000","last_peered":"2024-04-22T05:55:37.732937+0000","last_clean":"2024-04-22T05:55:37.732937+0000","last_became_active":"2024-04-22T05:50:47.172218+0000","last_became_peered":"2024-04-22T05:50:47.172218+0000","last_unstale":"2024-04-22T05:55:37.732937+0000","last_undegraded":"2024-04-22T05:55:37.732937+0000","last_fullsized":"2024-04-22T05:55:37.732937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5425","last_scrub_stamp":"2024-04-22T05:55:37.732870+0000","last_deep_scrub":"23'5425","last_deep_scrub_stamp":"2024-04-22T05:55:37.732870+0000","last_clean_scrub_stamp":"2024-04-22T05:55:37.732870+0000","objects_scrubbed":35,"log_size":5425,"log_dups_size":0,"ondisk_log_size":5425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:13:05.798101+0000","scrub_duration":0.59605697599999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138609688,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":5358,"num_write_kb":177059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:22.680694+0000","last_change":"2024-04-22T05:55:22.680782+0000","last_active":"2024-04-22T05:55:22.680694+0000","last_peered":"2024-04-22T05:55:22.680694+0000","last_clean":"2024-04-22T05:55:22.680694+0000","last_became_active":"2024-04-22T05:50:41.051259+0000","last_became_peered":"2024-04-22T05:50:41.051259+0000","last_unstale":"2024-04-22T05:55:22.680694+0000","last_undegraded":"2024-04-22T05:55:22.680694+0000","last_fullsized":"2024-04-22T05:55:22.680694+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:22.680676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:22.680676+0000","last_clean_scrub_stamp":"2024-04-22T05:55:22.680676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:05:34.545220+0000","scrub_duration":0.001356763,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"23'4981","reported_seq":5111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:29.485246+0000","last_change":"2024-04-22T05:55:29.485517+0000","last_active":"2024-04-22T05:55:29.485246+0000","last_peered":"2024-04-22T05:55:29.485246+0000","last_clean":"2024-04-22T05:55:29.485246+0000","last_became_active":"2024-04-22T05:50:47.119414+0000","last_became_peered":"2024-04-22T05:50:47.119414+0000","last_unstale":"2024-04-22T05:55:29.485246+0000","last_undegraded":"2024-04-22T05:55:29.485246+0000","last_fullsized":"2024-04-22T05:55:29.485246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4981","last_scrub_stamp":"2024-04-22T05:55:29.485120+0000","last_deep_scrub":"23'4981","last_deep_scrub_stamp":"2024-04-22T05:55:29.485120+0000","last_clean_scrub_stamp":"2024-04-22T05:55:29.485120+0000","objects_scrubbed":31,"log_size":4981,"log_dups_size":0,"ondisk_log_size":4981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:41:37.190926+0000","scrub_duration":0.48502272699999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124520800,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":4927,"num_write_kb":205619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:29.064418+0000","last_change":"2024-04-22T05:55:29.064619+0000","last_active":"2024-04-22T05:55:29.064418+0000","last_peered":"2024-04-22T05:55:29.064418+0000","last_clean":"2024-04-22T05:55:29.064418+0000","last_became_active":"2024-04-22T05:50:41.044865+0000","last_became_peered":"2024-04-22T05:50:41.044865+0000","last_unstale":"2024-04-22T05:55:29.064418+0000","last_undegraded":"2024-04-22T05:55:29.064418+0000","last_fullsized":"2024-04-22T05:55:29.064418+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:29.064368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:29.064368+0000","last_clean_scrub_stamp":"2024-04-22T05:55:29.064368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:20:39.181665+0000","scrub_duration":0.003329364,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"23'5463","reported_seq":5492,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:29.818270+0000","last_change":"2024-04-22T05:55:29.818555+0000","last_active":"2024-04-22T05:55:29.818270+0000","last_peered":"2024-04-22T05:55:29.818270+0000","last_clean":"2024-04-22T05:55:29.818270+0000","last_became_active":"2024-04-22T05:50:47.113405+0000","last_became_peered":"2024-04-22T05:50:47.113405+0000","last_unstale":"2024-04-22T05:55:29.818270+0000","last_undegraded":"2024-04-22T05:55:29.818270+0000","last_fullsized":"2024-04-22T05:55:29.818270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5463","last_scrub_stamp":"2024-04-22T05:55:29.818155+0000","last_deep_scrub":"23'5463","last_deep_scrub_stamp":"2024-04-22T05:55:29.818155+0000","last_clean_scrub_stamp":"2024-04-22T05:55:29.818155+0000","objects_scrubbed":32,"log_size":5463,"log_dups_size":0,"ondisk_log_size":5463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:24:10.903144+0000","scrub_duration":0.52685615100000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128647688,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5434,"num_write_kb":203340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:31.546917+0000","last_change":"2024-04-22T05:55:31.547161+0000","last_active":"2024-04-22T05:55:31.546917+0000","last_peered":"2024-04-22T05:55:31.546917+0000","last_clean":"2024-04-22T05:55:31.546917+0000","last_became_active":"2024-04-22T05:50:41.052349+0000","last_became_peered":"2024-04-22T05:50:41.052349+0000","last_unstale":"2024-04-22T05:55:31.546917+0000","last_undegraded":"2024-04-22T05:55:31.546917+0000","last_fullsized":"2024-04-22T05:55:31.546917+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T05:55:31.546866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T05:55:31.546866+0000","last_clean_scrub_stamp":"2024-04-22T05:55:31.546866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:06:34.832231+0000","scrub_duration":0.0034443590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"23'119","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:28.094886+0000","last_change":"2024-04-22T05:55:28.095221+0000","last_active":"2024-04-22T05:55:28.094886+0000","last_peered":"2024-04-22T05:55:28.094886+0000","last_clean":"2024-04-22T05:55:28.094886+0000","last_became_active":"2024-04-22T05:50:40.046329+0000","last_became_peered":"2024-04-22T05:50:40.046329+0000","last_unstale":"2024-04-22T05:55:28.094886+0000","last_undegraded":"2024-04-22T05:55:28.094886+0000","last_fullsized":"2024-04-22T05:55:28.094886+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'119","last_scrub_stamp":"2024-04-22T05:55:28.094771+0000","last_deep_scrub":"23'119","last_deep_scrub_stamp":"2024-04-22T05:55:28.094771+0000","last_clean_scrub_stamp":"2024-04-22T05:55:28.094771+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:17:52.246384+0000","scrub_duration":0.011123787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"23'3960","reported_seq":3991,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:23.486878+0000","last_change":"2024-04-22T05:55:23.486951+0000","last_active":"2024-04-22T05:55:23.486878+0000","last_peered":"2024-04-22T05:55:23.486878+0000","last_clean":"2024-04-22T05:55:23.486878+0000","last_became_active":"2024-04-22T05:50:47.115449+0000","last_became_peered":"2024-04-22T05:50:47.115449+0000","last_unstale":"2024-04-22T05:55:23.486878+0000","last_undegraded":"2024-04-22T05:55:23.486878+0000","last_fullsized":"2024-04-22T05:55:23.486878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3960","last_scrub_stamp":"2024-04-22T05:55:23.486858+0000","last_deep_scrub":"23'3960","last_deep_scrub_stamp":"2024-04-22T05:55:23.486858+0000","last_clean_scrub_stamp":"2024-04-22T05:55:23.486858+0000","objects_scrubbed":28,"log_size":3960,"log_dups_size":0,"ondisk_log_size":3960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:40.380402+0000","scrub_duration":0.25562303400000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":3867,"num_write_kb":173187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"23'10268","reported_seq":10411,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:27.499815+0000","last_change":"2024-04-22T05:55:27.499971+0000","last_active":"2024-04-22T05:55:27.499815+0000","last_peered":"2024-04-22T05:55:27.499815+0000","last_clean":"2024-04-22T05:55:27.499815+0000","last_became_active":"2024-04-22T05:50:47.173440+0000","last_became_peered":"2024-04-22T05:50:47.173440+0000","last_unstale":"2024-04-22T05:55:27.499815+0000","last_undegraded":"2024-04-22T05:55:27.499815+0000","last_fullsized":"2024-04-22T05:55:27.499815+0000","mapping_epoch":19,"log_start":"21'200","ondisk_log_start":"21'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10268","last_scrub_stamp":"2024-04-22T05:55:27.499781+0000","last_deep_scrub":"23'10268","last_deep_scrub_stamp":"2024-04-22T05:55:27.499781+0000","last_clean_scrub_stamp":"2024-04-22T05:55:27.499781+0000","objects_scrubbed":44,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:53:27.639500+0000","scrub_duration":0.76434783799999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":381,"num_read_kb":346,"num_write":8425,"num_write_kb":237093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"23'5065","reported_seq":5093,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:30.343397+0000","last_change":"2024-04-22T05:55:30.343707+0000","last_active":"2024-04-22T05:55:30.343397+0000","last_peered":"2024-04-22T05:55:30.343397+0000","last_clean":"2024-04-22T05:55:30.343397+0000","last_became_active":"2024-04-22T05:50:47.120541+0000","last_became_peered":"2024-04-22T05:50:47.120541+0000","last_unstale":"2024-04-22T05:55:30.343397+0000","last_undegraded":"2024-04-22T05:55:30.343397+0000","last_fullsized":"2024-04-22T05:55:30.343397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5065","last_scrub_stamp":"2024-04-22T05:55:30.343343+0000","last_deep_scrub":"23'5065","last_deep_scrub_stamp":"2024-04-22T05:55:30.343343+0000","last_clean_scrub_stamp":"2024-04-22T05:55:30.343343+0000","objects_scrubbed":29,"log_size":5065,"log_dups_size":0,"ondisk_log_size":5065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:46:22.673675+0000","scrub_duration":0.51344110700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5009,"num_write_kb":193539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"23'4738","reported_seq":4767,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:30.682420+0000","last_change":"2024-04-22T05:55:30.682612+0000","last_active":"2024-04-22T05:55:30.682420+0000","last_peered":"2024-04-22T05:55:30.682420+0000","last_clean":"2024-04-22T05:55:30.682420+0000","last_became_active":"2024-04-22T05:50:47.123115+0000","last_became_peered":"2024-04-22T05:50:47.123115+0000","last_unstale":"2024-04-22T05:55:30.682420+0000","last_undegraded":"2024-04-22T05:55:30.682420+0000","last_fullsized":"2024-04-22T05:55:30.682420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4738","last_scrub_stamp":"2024-04-22T05:55:30.682374+0000","last_deep_scrub":"23'4738","last_deep_scrub_stamp":"2024-04-22T05:55:30.682374+0000","last_clean_scrub_stamp":"2024-04-22T05:55:30.682374+0000","objects_scrubbed":24,"log_size":4738,"log_dups_size":0,"ondisk_log_size":4738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:37:19.251975+0000","scrub_duration":0.36940600099999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600592,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":4720,"num_write_kb":218508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"23'4402","reported_seq":4432,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:32.933610+0000","last_change":"2024-04-22T05:55:32.933949+0000","last_active":"2024-04-22T05:55:32.933610+0000","last_peered":"2024-04-22T05:55:32.933610+0000","last_clean":"2024-04-22T05:55:32.933610+0000","last_became_active":"2024-04-22T05:50:47.114834+0000","last_became_peered":"2024-04-22T05:50:47.114834+0000","last_unstale":"2024-04-22T05:55:32.933610+0000","last_undegraded":"2024-04-22T05:55:32.933610+0000","last_fullsized":"2024-04-22T05:55:32.933610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4402","last_scrub_stamp":"2024-04-22T05:55:32.933553+0000","last_deep_scrub":"23'4402","last_deep_scrub_stamp":"2024-04-22T05:55:32.933553+0000","last_clean_scrub_stamp":"2024-04-22T05:55:32.933553+0000","objects_scrubbed":24,"log_size":4402,"log_dups_size":0,"ondisk_log_size":4402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:23:14.667949+0000","scrub_duration":0.35749684700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":4338,"num_write_kb":218182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'3579","reported_seq":3607,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:41.289292+0000","last_change":"2024-04-22T05:55:41.289526+0000","last_active":"2024-04-22T05:55:41.289292+0000","last_peered":"2024-04-22T05:55:41.289292+0000","last_clean":"2024-04-22T05:55:41.289292+0000","last_became_active":"2024-04-22T05:50:47.113260+0000","last_became_peered":"2024-04-22T05:50:47.113260+0000","last_unstale":"2024-04-22T05:55:41.289292+0000","last_undegraded":"2024-04-22T05:55:41.289292+0000","last_fullsized":"2024-04-22T05:55:41.289292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3579","last_scrub_stamp":"2024-04-22T05:55:41.289242+0000","last_deep_scrub":"23'3579","last_deep_scrub_stamp":"2024-04-22T05:55:41.289242+0000","last_clean_scrub_stamp":"2024-04-22T05:55:41.289242+0000","objects_scrubbed":25,"log_size":3579,"log_dups_size":0,"ondisk_log_size":3579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:50:00.346236+0000","scrub_duration":0.38064881099999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104202240,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3525,"num_write_kb":160132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'3652","reported_seq":3770,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:42.247106+0000","last_change":"2024-04-22T05:55:42.247333+0000","last_active":"2024-04-22T05:55:42.247106+0000","last_peered":"2024-04-22T05:55:42.247106+0000","last_clean":"2024-04-22T05:55:42.247106+0000","last_became_active":"2024-04-22T05:50:47.122822+0000","last_became_peered":"2024-04-22T05:50:47.122822+0000","last_unstale":"2024-04-22T05:55:42.247106+0000","last_undegraded":"2024-04-22T05:55:42.247106+0000","last_fullsized":"2024-04-22T05:55:42.247106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3652","last_scrub_stamp":"2024-04-22T05:55:42.247060+0000","last_deep_scrub":"23'3652","last_deep_scrub_stamp":"2024-04-22T05:55:42.247060+0000","last_clean_scrub_stamp":"2024-04-22T05:55:42.247060+0000","objects_scrubbed":20,"log_size":3652,"log_dups_size":0,"ondisk_log_size":3652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:17:57.962138+0000","scrub_duration":0.33332855900000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":300,"num_read_kb":235,"num_write":3883,"num_write_kb":108782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'4364","reported_seq":4416,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:45.318042+0000","last_change":"2024-04-22T05:55:45.318247+0000","last_active":"2024-04-22T05:55:45.318042+0000","last_peered":"2024-04-22T05:55:45.318042+0000","last_clean":"2024-04-22T05:55:45.318042+0000","last_became_active":"2024-04-22T05:50:47.117734+0000","last_became_peered":"2024-04-22T05:50:47.117734+0000","last_unstale":"2024-04-22T05:55:45.318042+0000","last_undegraded":"2024-04-22T05:55:45.318042+0000","last_fullsized":"2024-04-22T05:55:45.318042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4364","last_scrub_stamp":"2024-04-22T05:55:45.317987+0000","last_deep_scrub":"23'4364","last_deep_scrub_stamp":"2024-04-22T05:55:45.317987+0000","last_clean_scrub_stamp":"2024-04-22T05:55:45.317987+0000","objects_scrubbed":33,"log_size":4364,"log_dups_size":0,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:01.040022+0000","scrub_duration":0.45398512600000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126091776,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":92,"num_read_kb":134,"num_write":4330,"num_write_kb":222405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"23'3218","reported_seq":3244,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:37.140959+0000","last_change":"2024-04-22T05:55:37.141270+0000","last_active":"2024-04-22T05:55:37.140959+0000","last_peered":"2024-04-22T05:55:37.140959+0000","last_clean":"2024-04-22T05:55:37.140959+0000","last_became_active":"2024-04-22T05:50:47.124711+0000","last_became_peered":"2024-04-22T05:50:47.124711+0000","last_unstale":"2024-04-22T05:55:37.140959+0000","last_undegraded":"2024-04-22T05:55:37.140959+0000","last_fullsized":"2024-04-22T05:55:37.140959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3218","last_scrub_stamp":"2024-04-22T05:55:37.140851+0000","last_deep_scrub":"23'3218","last_deep_scrub_stamp":"2024-04-22T05:55:37.140851+0000","last_clean_scrub_stamp":"2024-04-22T05:55:37.140851+0000","objects_scrubbed":21,"log_size":3218,"log_dups_size":0,"ondisk_log_size":3218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:07:41.912754+0000","scrub_duration":0.27221202,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3149,"num_write_kb":120072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"23'3721","reported_seq":4053,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:47.196164+0000","last_change":"2024-04-22T05:55:47.196460+0000","last_active":"2024-04-22T05:55:47.196164+0000","last_peered":"2024-04-22T05:55:47.196164+0000","last_clean":"2024-04-22T05:55:47.196164+0000","last_became_active":"2024-04-22T05:50:47.115535+0000","last_became_peered":"2024-04-22T05:50:47.115535+0000","last_unstale":"2024-04-22T05:55:47.196164+0000","last_undegraded":"2024-04-22T05:55:47.196164+0000","last_fullsized":"2024-04-22T05:55:47.196164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3721","last_scrub_stamp":"2024-04-22T05:55:47.196104+0000","last_deep_scrub":"23'3721","last_deep_scrub_stamp":"2024-04-22T05:55:47.196104+0000","last_clean_scrub_stamp":"2024-04-22T05:55:47.196104+0000","objects_scrubbed":24,"log_size":3721,"log_dups_size":0,"ondisk_log_size":3721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:43:00.006048+0000","scrub_duration":0.34043821400000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92406272,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1026,"num_read_kb":919,"num_write":3912,"num_write_kb":173447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"23'3456","reported_seq":3485,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:23.312857+0000","last_change":"2024-04-22T05:55:23.312857+0000","last_active":"2024-04-22T05:55:23.312857+0000","last_peered":"2024-04-22T05:55:23.312857+0000","last_clean":"2024-04-22T05:55:23.312857+0000","last_became_active":"2024-04-22T05:50:47.129345+0000","last_became_peered":"2024-04-22T05:50:47.129345+0000","last_unstale":"2024-04-22T05:55:23.312857+0000","last_undegraded":"2024-04-22T05:55:23.312857+0000","last_fullsized":"2024-04-22T05:55:23.312857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3456","last_scrub_stamp":"2024-04-22T05:55:23.312622+0000","last_deep_scrub":"23'3456","last_deep_scrub_stamp":"2024-04-22T05:55:23.312622+0000","last_clean_scrub_stamp":"2024-04-22T05:55:23.312622+0000","objects_scrubbed":18,"log_size":3456,"log_dups_size":0,"ondisk_log_size":3456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T06:34:38.998900+0000","scrub_duration":0.21609183200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":71,"num_write":3499,"num_write_kb":148421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"23'6873","reported_seq":6900,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T05:55:52.407660+0000","last_change":"2024-04-22T05:55:52.407866+0000","last_active":"2024-04-22T05:55:52.407660+0000","last_peered":"2024-04-22T05:55:52.407660+0000","last_clean":"2024-04-22T05:55:52.407660+0000","last_became_active":"2024-04-22T05:50:47.126563+0000","last_became_peered":"2024-04-22T05:50:47.126563+0000","last_unstale":"2024-04-22T05:55:52.407660+0000","last_undegraded":"2024-04-22T05:55:52.407660+0000","last_fullsized":"2024-04-22T05:55:52.407660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6873","last_scrub_stamp":"2024-04-22T05:55:52.407618+0000","last_deep_scrub":"23'6873","last_deep_scrub_stamp":"2024-04-22T05:55:52.407618+0000","last_clean_scrub_stamp":"2024-04-22T05:55:52.407618+0000","objects_scrubbed":24,"log_size":6873,"log_dups_size":0,"ondisk_log_size":6873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:01:21.666696+0000","scrub_duration":0.461029523,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1,"num_read_kb":64,"num_write":6824,"num_write_kb":157279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3567505656,"num_objects":877,"num_object_clones":0,"num_object_copies":1754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":877,"num_whiteouts":0,"num_read":2560,"num_read_kb":2576,"num_write":157144,"num_write_kb":5626241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6796083200,"data_stored":6833661048,"data_compressed":194275682,"data_compressed_allocated":202047488,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":159417,"ondisk_log_size":159417,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11158,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":14,"seq":60129542211,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591960,"kb_used_data":408272,"kb_used_omap":1,"kb_used_meta":183678,"kb_avail":93779880,"statfs":{"total":96636764160,"available":96030597120,"internally_reserved":0,"allocated":418070528,"data_stored":419212919,"data_compressed":12754442,"data_compressed_allocated":13279232,"data_compressed_original":14893056,"omap_allocated":1594,"internal_metadata":188086726},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.83299999999999996,"15min":0.78300000000000003},"min":{"1min":0.46800000000000003,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":1.0289999999999999,"5min":20.890000000000001,"15min":20.890000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.64700000000000002,"15min":0.66900000000000004},"min":{"1min":0.49199999999999999,"5min":0.185,"15min":0.185},"max":{"1min":1.1120000000000001,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.83399999999999996}]},{"osd":1,"last update":"Mon Apr 22 05:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67100000000000004,"15min":0.67400000000000004},"min":{"1min":0.41699999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.89700000000000002,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.66000000000000003,"15min":0.60399999999999998},"min":{"1min":0.313,"5min":0.221,"15min":0.221},"max":{"1min":1.1060000000000001,"5min":1.218,"15min":1.218},"last":1.0569999999999999}]},{"osd":3,"last update":"Mon Apr 22 05:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.71199999999999997,"15min":0.85399999999999998},"min":{"1min":0.47099999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.0489999999999999,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79200000000000004,"15min":0.878},"min":{"1min":0.54400000000000004,"5min":0.246,"15min":0.246},"max":{"1min":1.1850000000000001,"5min":3.27,"15min":3.27},"last":1.0149999999999999}]},{"osd":4,"last update":"Mon Apr 22 05:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.84299999999999997,"15min":0.81100000000000005},"min":{"1min":0.59499999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.1890000000000001,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.097},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.90600000000000003,"15min":0.86899999999999999},"min":{"1min":0.67200000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.351,"5min":1.54,"15min":1.54},"last":1.153}]},{"osd":5,"last update":"Mon Apr 22 05:55:51 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.86799999999999999,"15min":0.85899999999999999},"min":{"1min":0.745,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.054,"5min":0.92700000000000005,"15min":0.90400000000000003},"min":{"1min":0.57899999999999996,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.569,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.3720000000000001}]},{"osd":6,"last update":"Mon Apr 22 05:55:51 2024","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.94099999999999995,"15min":0.95899999999999996},"min":{"1min":0.55900000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.5249999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":1.218},{"interface":"front","average":{"1min":1.044,"5min":0.95599999999999996,"15min":0.97799999999999998},"min":{"1min":0.626,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.3480000000000001,"5min":1.651,"15min":1.651},"last":1.4390000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:55:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.98499999999999999,"15min":0.98899999999999999},"min":{"1min":0.64300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":1.581,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.343},{"interface":"front","average":{"1min":1.081,"5min":1.111,"15min":1.026},"min":{"1min":0.78400000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.667,"5min":17.995999999999999,"15min":17.995999999999999},"last":1.405}]}]},{"osd":6,"up_from":14,"seq":60129542214,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418588,"kb_used_data":311188,"kb_used_omap":1,"kb_used_meta":107390,"kb_avail":93953252,"statfs":{"total":96636764160,"available":96208130048,"internally_reserved":0,"allocated":318656512,"data_stored":318798411,"data_compressed":4811634,"data_compressed_allocated":5001216,"data_compressed_original":5484544,"omap_allocated":1594,"internal_metadata":109967814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":1.2629999999999999,"15min":1.7569999999999999},"min":{"1min":0.50700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.196,"5min":34.100000000000001,"15min":34.100000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":1.0720000000000001,"15min":0.85599999999999998},"min":{"1min":0.56100000000000005,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.123,"5min":30.306000000000001,"15min":30.306000000000001},"last":0.76300000000000001}]},{"osd":1,"last update":"Mon Apr 22 05:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.84199999999999997,"15min":0.755},"min":{"1min":0.51000000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.2470000000000001,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.133},{"interface":"front","average":{"1min":1.002,"5min":0.86499999999999999,"15min":0.79300000000000004},"min":{"1min":0.67000000000000004,"5min":0.249,"15min":0.249},"max":{"1min":1.3089999999999999,"5min":1.74,"15min":1.74},"last":1.101}]},{"osd":2,"last update":"Mon Apr 22 05:55:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.92300000000000004,"15min":0.84899999999999998},"min":{"1min":0.69499999999999995,"5min":0.216,"15min":0.216},"max":{"1min":1.3560000000000001,"5min":2.488,"15min":2.488},"last":1.014},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.95699999999999996,"15min":0.89700000000000002},"min":{"1min":0.60899999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.2729999999999999,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":0.94699999999999995}]},{"osd":3,"last update":"Mon Apr 22 05:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.85499999999999998,"15min":0.86299999999999999},"min":{"1min":0.72799999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.129,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0309999999999999,"15min":0.96299999999999997},"min":{"1min":0.68300000000000005,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1910000000000001,"5min":13.539,"15min":13.539},"last":0.90800000000000003}]},{"osd":4,"last update":"Mon Apr 22 05:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88400000000000001,"15min":0.92500000000000004},"min":{"1min":0.64200000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.119,"5min":3.0270000000000001,"15min":3.0270000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.90800000000000003,"15min":0.875},"min":{"1min":0.77500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.22,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.877}]},{"osd":5,"last update":"Mon Apr 22 05:55:49 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.046,"15min":1.212},"min":{"1min":0.74299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.284,"5min":11.922000000000001,"15min":11.922000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0109999999999999,"15min":0.999},"min":{"1min":0.71899999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2929999999999999,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.80900000000000005}]},{"osd":7,"last update":"Mon Apr 22 05:55:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.0620000000000001,"15min":1.006},"min":{"1min":0.997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3169999999999999,"5min":2.129,"15min":2.129},"last":1.081},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.095,"15min":1.099},"min":{"1min":0.73699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.298,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.052}]}]},{"osd":5,"up_from":14,"seq":60129542212,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1090860,"kb_used_data":843236,"kb_used_omap":1,"kb_used_meta":247614,"kb_avail":93280980,"statfs":{"total":96636764160,"available":95519723520,"internally_reserved":0,"allocated":863473664,"data_stored":869120975,"data_compressed":21037680,"data_compressed_allocated":21954560,"data_compressed_original":28217344,"omap_allocated":1594,"internal_metadata":253557190},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.80600000000000005,"15min":0.70399999999999996},"min":{"1min":0.58399999999999996,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.2689999999999999,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.018,"5min":0.89300000000000002,"15min":0.91600000000000004},"min":{"1min":0.69199999999999995,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.337,"5min":4.6390000000000002,"15min":4.6390000000000002},"last":1.5449999999999999}]},{"osd":1,"last update":"Mon Apr 22 05:55:50 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.85699999999999998,"15min":0.73699999999999999},"min":{"1min":0.71099999999999997,"5min":0.253,"15min":0.253},"max":{"1min":1.258,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.92400000000000004,"15min":1.1539999999999999},"min":{"1min":0.68700000000000006,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.0720000000000001,"5min":8.202,"15min":8.202},"last":1.0780000000000001}]},{"osd":2,"last update":"Mon Apr 22 05:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.875,"15min":0.80200000000000005},"min":{"1min":0.65900000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4119999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.175},{"interface":"front","average":{"1min":1.1259999999999999,"5min":0.96699999999999997,"15min":0.85899999999999999},"min":{"1min":0.63900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.397,"5min":1.673,"15min":1.673},"last":0.99299999999999999}]},{"osd":3,"last update":"Mon Apr 22 05:55:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.5880000000000001,"15min":2.6800000000000002},"min":{"1min":0.73099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.4179999999999999,"5min":44.186999999999998,"15min":44.186999999999998},"last":1.4279999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.0229999999999999,"15min":1.179},"min":{"1min":0.70999999999999996,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.224,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":1.1930000000000001}]},{"osd":4,"last update":"Mon Apr 22 05:55:50 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.1279999999999999,"15min":1.3120000000000001},"min":{"1min":0.61899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.77,"5min":11.438000000000001,"15min":11.438000000000001},"last":1.4990000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.98499999999999999,"15min":0.95999999999999996},"min":{"1min":0.66100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.119,"5min":2.181,"15min":2.181},"last":1.0509999999999999}]},{"osd":6,"last update":"Mon Apr 22 05:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94499999999999995,"15min":0.93500000000000005},"min":{"1min":0.69399999999999995,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.331,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.3899999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.99299999999999999,"15min":1.3},"min":{"1min":0.67900000000000005,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.1499999999999999,"5min":11.474,"15min":11.474},"last":0.95899999999999996}]},{"osd":7,"last update":"Mon Apr 22 05:55:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.145,"15min":1.0940000000000001},"min":{"1min":0.95099999999999996,"5min":0.372,"15min":0.372},"max":{"1min":1.508,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.8280000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.1020000000000001,"15min":1.155},"min":{"1min":0.89200000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4339999999999999,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":1.464}]}]},{"osd":4,"up_from":14,"seq":60129542213,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1920752,"kb_used_data":1416424,"kb_used_omap":1,"kb_used_meta":504318,"kb_avail":92451088,"statfs":{"total":96636764160,"available":94669914112,"internally_reserved":0,"allocated":1450418176,"data_stored":1463127323,"data_compressed":51243088,"data_compressed_allocated":53432320,"data_compressed_original":67096576,"omap_allocated":1594,"internal_metadata":516422086},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:55:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.98399999999999999,"15min":0.78000000000000003},"min":{"1min":0.64500000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.327,"5min":12.618,"15min":12.618},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.0609999999999999,"15min":0.92700000000000005},"min":{"1min":0.73899999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.5029999999999999,"5min":12.548,"15min":12.548},"last":1.1719999999999999}]},{"osd":1,"last update":"Mon Apr 22 05:55:46 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.77900000000000003,"15min":0.76400000000000001},"min":{"1min":0.627,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.1279999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.877},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.024,"15min":0.83899999999999997},"min":{"1min":0.69999999999999996,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2789999999999999,"5min":12.449999999999999,"15min":12.449999999999999},"last":0.98099999999999998}]},{"osd":2,"last update":"Mon Apr 22 05:55:46 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.82399999999999995,"15min":0.80900000000000005},"min":{"1min":0.68799999999999994,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.3540000000000001,"5min":2.226,"15min":2.226},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.093,"15min":0.94299999999999995},"min":{"1min":0.83899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.3999999999999999,"5min":12.673,"15min":12.673},"last":1.127}]},{"osd":3,"last update":"Mon Apr 22 05:55:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.93700000000000006,"15min":0.89500000000000002},"min":{"1min":0.67700000000000005,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.4159999999999999,"5min":3.5329999999999999,"15min":3.5329999999999999},"last":0.749},{"interface":"front","average":{"1min":1.109,"5min":0.99199999999999999,"15min":0.96199999999999997},"min":{"1min":0.76400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.448,"5min":2.254,"15min":2.254},"last":0.93400000000000005}]},{"osd":5,"last update":"Mon Apr 22 05:55:46 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.133,"15min":1.0449999999999999},"min":{"1min":0.78200000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.607,"5min":12.721,"15min":12.721},"last":1.042},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.93500000000000005,"15min":0.96199999999999997},"min":{"1min":0.64400000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.3640000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.81699999999999995}]},{"osd":6,"last update":"Mon Apr 22 05:55:46 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.224,"15min":1.05},"min":{"1min":0.81899999999999995,"5min":0.312,"15min":0.312},"max":{"1min":1.3779999999999999,"5min":12.765000000000001,"15min":12.765000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.1160000000000001,"15min":1.097},"min":{"1min":0.99199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.669,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.085}]},{"osd":7,"last update":"Mon Apr 22 05:55:46 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.0509999999999999,"15min":1.0609999999999999},"min":{"1min":0.85299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.5600000000000001,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.135,"15min":1.077},"min":{"1min":1.022,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.603,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":0.96599999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542214,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1378076,"kb_used_data":1018004,"kb_used_omap":1,"kb_used_meta":360062,"kb_avail":92993764,"statfs":{"total":96636764160,"available":95225614336,"internally_reserved":0,"allocated":1042436096,"data_stored":1047505319,"data_compressed":29839922,"data_compressed_allocated":30982144,"data_compressed_original":36835328,"omap_allocated":1593,"internal_metadata":368703943},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Apr 22 05:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.67600000000000005,"15min":0.57399999999999995},"min":{"1min":0.23699999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":4.4429999999999996,"5min":4.4429999999999996,"15min":4.4429999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70699999999999996,"15min":0.79500000000000004},"min":{"1min":0.312,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.93400000000000005,"5min":4.7489999999999997,"15min":4.7489999999999997},"last":0.59599999999999997}]},{"osd":2,"last update":"Mon Apr 22 05:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.75600000000000001,"15min":0.64800000000000002},"min":{"1min":0.28499999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":4.3319999999999999,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.84399999999999997,"15min":0.89400000000000002},"min":{"1min":0.26700000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.085,"5min":4.7880000000000003,"15min":4.7880000000000003},"last":0.85399999999999998}]},{"osd":3,"last update":"Mon Apr 22 05:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.73599999999999999,"15min":0.66300000000000003},"min":{"1min":0.32000000000000001,"5min":0.219,"15min":0.219},"max":{"1min":4.8789999999999996,"5min":4.8789999999999996,"15min":4.8789999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.083,"5min":0.86099999999999999,"15min":0.78500000000000003},"min":{"1min":0.437,"5min":0.307,"15min":0.307},"max":{"1min":4.6840000000000002,"5min":4.6840000000000002,"15min":4.6840000000000002},"last":0.98899999999999999}]},{"osd":4,"last update":"Mon Apr 22 05:55:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.073,"15min":1.1060000000000001},"min":{"1min":0.54900000000000004,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":8.0530000000000008,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.248,"5min":0.98399999999999999,"15min":0.92700000000000005},"min":{"1min":0.752,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.8049999999999997,"5min":4.8049999999999997,"15min":4.8049999999999997},"last":1.177}]},{"osd":5,"last update":"Mon Apr 22 05:55:45 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.0169999999999999,"15min":0.95499999999999996},"min":{"1min":0.57199999999999995,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":4.5730000000000004,"5min":4.5730000000000004,"15min":4.5730000000000004},"last":1.052},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.0409999999999999,"15min":1.111},"min":{"1min":0.63800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":5.0659999999999998,"5min":5.0659999999999998,"15min":5.0659999999999998},"last":1.0780000000000001}]},{"osd":6,"last update":"Mon Apr 22 05:55:49 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.0449999999999999,"15min":0.91700000000000004},"min":{"1min":0.61399999999999999,"5min":0.374,"15min":0.374},"max":{"1min":4.6379999999999999,"5min":4.6379999999999999,"15min":4.6379999999999999},"last":1.202},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.028,"15min":0.92200000000000004},"min":{"1min":0.57699999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":5.016,"5min":5.016,"15min":5.016},"last":0.92400000000000004}]},{"osd":7,"last update":"Mon Apr 22 05:55:45 2024","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.2350000000000001,"15min":1.393},"min":{"1min":0.67700000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.851,"5min":8.6839999999999993,"15min":8.6839999999999993},"last":1.226},{"interface":"front","average":{"1min":1.353,"5min":1.212,"15min":1.256},"min":{"1min":0.54000000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.9539999999999997,"5min":4.9539999999999997,"15min":4.9539999999999997},"last":1.0209999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574918,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1333656,"kb_used_data":959696,"kb_used_omap":1,"kb_used_meta":373950,"kb_avail":93038184,"statfs":{"total":96636764160,"available":95271100416,"internally_reserved":0,"allocated":982728704,"data_stored":987914375,"data_compressed":23928438,"data_compressed_allocated":24997888,"data_compressed_original":30998528,"omap_allocated":1589,"internal_metadata":382925259},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:55:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1.339,"15min":1.0589999999999999},"min":{"1min":0.60599999999999998,"5min":0.224,"15min":0.224},"max":{"1min":1.147,"5min":46.598999999999997,"15min":46.598999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.91300000000000003,"15min":1.2310000000000001},"min":{"1min":0.53300000000000003,"5min":0.216,"15min":0.216},"max":{"1min":1.149,"5min":15.144,"15min":15.144},"last":0.66600000000000004}]},{"osd":1,"last update":"Mon Apr 22 05:55:48 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.83999999999999997,"15min":0.78600000000000003},"min":{"1min":0.56699999999999995,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.2749999999999999,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.042,"5min":0.94699999999999995,"15min":0.84399999999999997},"min":{"1min":0.73699999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3839999999999999,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.171}]},{"osd":2,"last update":"Mon Apr 22 05:55:48 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.91000000000000003,"15min":0.876},"min":{"1min":0.66100000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.196,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.99099999999999999,"15min":0.88700000000000001},"min":{"1min":0.80100000000000005,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.248,"5min":3.387,"15min":3.387},"last":1.133}]},{"osd":3,"last update":"Mon Apr 22 05:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.036,"15min":0.871},"min":{"1min":0.623,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2869999999999999,"5min":7.1699999999999999,"15min":7.1699999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.022,"5min":0.94199999999999995,"15min":0.84299999999999997},"min":{"1min":0.746,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3080000000000001,"5min":3.6299999999999999,"15min":3.6299999999999999},"last":1.0820000000000001}]},{"osd":4,"last update":"Mon Apr 22 05:55:48 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.2110000000000001,"15min":1.2749999999999999},"min":{"1min":0.64700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.621,"5min":9.6620000000000008,"15min":9.6620000000000008},"last":1.222},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.96899999999999997,"15min":0.90900000000000003},"min":{"1min":0.69699999999999995,"5min":0.252,"15min":0.252},"max":{"1min":1.3009999999999999,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":0.871}]},{"osd":5,"last update":"Mon Apr 22 05:55:48 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.97199999999999998,"15min":0.90500000000000003},"min":{"1min":0.70499999999999996,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.208,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2110000000000001,"15min":1.012},"min":{"1min":0.73799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.3759999999999999,"5min":7.2430000000000003,"15min":7.2430000000000003},"last":0.94899999999999995}]},{"osd":6,"last update":"Mon Apr 22 05:55:48 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.0820000000000001,"15min":0.96999999999999997},"min":{"1min":0.80600000000000005,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.6200000000000001,"5min":3.5939999999999999,"15min":3.5939999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2869999999999999,"15min":1.079},"min":{"1min":0.76600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.6859999999999999,"5min":7.2960000000000003,"15min":7.2960000000000003},"last":1.272}]}]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1412656,"kb_used_data":1016552,"kb_used_omap":1,"kb_used_meta":396094,"kb_avail":92959184,"statfs":{"total":96636764160,"available":95190204416,"internally_reserved":0,"allocated":1040949248,"data_stored":1044391251,"data_compressed":24230251,"data_compressed_allocated":25128960,"data_compressed_original":29396992,"omap_allocated":1592,"internal_metadata":405600712},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65100000000000002,"15min":0.81100000000000005},"min":{"1min":0.42999999999999999,"5min":0.191,"15min":0.191},"max":{"1min":1.052,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.139,"15min":2.125},"min":{"1min":0.32100000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.24,"5min":22.721,"15min":22.721},"last":0.68999999999999995}]},{"osd":1,"last update":"Mon Apr 22 05:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77100000000000002,"15min":0.93899999999999995},"min":{"1min":0.41599999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.036,"5min":9.6560000000000006,"15min":9.6560000000000006},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.69499999999999995,"15min":0.67500000000000004},"min":{"1min":0.47099999999999997,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.073,"5min":2.028,"15min":2.028},"last":0.73999999999999999}]},{"osd":2,"last update":"Mon Apr 22 05:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":1.107,"15min":1.5549999999999999},"min":{"1min":0.43099999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.0609999999999999,"5min":22.763999999999999,"15min":22.763999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.88300000000000001,"15min":1.0720000000000001},"min":{"1min":0.51200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.179,"5min":9.7739999999999991,"15min":9.7739999999999991},"last":0.79400000000000004}]},{"osd":4,"last update":"Mon Apr 22 05:55:45 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.538,"15min":1.825},"min":{"1min":0.78500000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4079999999999999,"5min":27.384,"15min":27.384},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.3640000000000001,"15min":1.8620000000000001},"min":{"1min":0.63200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3500000000000001,"5min":22.824000000000002,"15min":22.824000000000002},"last":1.0309999999999999}]},{"osd":5,"last update":"Mon Apr 22 05:55:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.4810000000000001,"15min":2.2280000000000002},"min":{"1min":0.67600000000000005,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.9970000000000001,"5min":25.565999999999999,"15min":25.565999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.095,"5min":1.024,"15min":0.95199999999999996},"min":{"1min":0.81799999999999995,"5min":0.378,"15min":0.378},"max":{"1min":1.5069999999999999,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":1.095}]},{"osd":6,"last update":"Mon Apr 22 05:55:45 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.4359999999999999,"15min":1.8879999999999999},"min":{"1min":0.72199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4650000000000001,"5min":22.794,"15min":22.794},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.1379999999999999,"15min":1.3120000000000001},"min":{"1min":0.871,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.712,"5min":9.9559999999999995,"15min":9.9559999999999995},"last":1.1220000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:55:45 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.9510000000000001,"15min":3.0059999999999998},"min":{"1min":0.65400000000000003,"5min":0.434,"15min":0.434},"max":{"1min":1.8839999999999999,"5min":36.770000000000003,"15min":36.770000000000003},"last":1.175},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3919999999999999,"15min":1.4019999999999999},"min":{"1min":0.75800000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.206,"5min":16.239000000000001,"15min":16.239000000000001},"last":1.2170000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":993392,"kb_used_data":665384,"kb_used_omap":1,"kb_used_meta":327998,"kb_avail":93378448,"statfs":{"total":96636764160,"available":95619530752,"internally_reserved":0,"allocated":681353216,"data_stored":685339487,"data_compressed":26441385,"data_compressed_allocated":27336704,"data_compressed_original":31993856,"omap_allocated":1594,"internal_metadata":335870406},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 05:55:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.66000000000000003,"15min":0.70499999999999996},"min":{"1min":0.36299999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.90800000000000003,"5min":3.9249999999999998,"15min":3.9249999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.69799999999999995,"15min":0.66700000000000004},"min":{"1min":0.25900000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.0109999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.0169999999999999}]},{"osd":2,"last update":"Mon Apr 22 05:55:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.41299999999999998,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.90100000000000002,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.75700000000000001,"15min":0.64100000000000001},"min":{"1min":0.39200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":1.1459999999999999,"5min":3.871,"15min":3.871},"last":1.244}]},{"osd":3,"last update":"Mon Apr 22 05:55:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.749,"15min":0.70299999999999996},"min":{"1min":0.43099999999999999,"5min":0.252,"15min":0.252},"max":{"1min":1.155,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.84199999999999997,"15min":0.85399999999999998},"min":{"1min":0.29799999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4239999999999999,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.76500000000000001}]},{"osd":4,"last update":"Mon Apr 22 05:55:48 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0249999999999999,"15min":1.113},"min":{"1min":0.57299999999999995,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3009999999999999,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.288},{"interface":"front","average":{"1min":1.131,"5min":1.0609999999999999,"15min":1.0489999999999999},"min":{"1min":0.77800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.425,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":0.93899999999999995}]},{"osd":5,"last update":"Mon Apr 22 05:55:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.028,"15min":0.95999999999999996},"min":{"1min":0.83299999999999996,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4239999999999999,"5min":3.8460000000000001,"15min":3.8460000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.0289999999999999,"15min":0.98299999999999998},"min":{"1min":0.64300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.641,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":1.198}]},{"osd":6,"last update":"Mon Apr 22 05:55:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.0369999999999999,"15min":0.94299999999999995},"min":{"1min":0.60699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.5820000000000001,"5min":2.9209999999999998,"15min":2.9209999999999998},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.034,"15min":0.97299999999999998},"min":{"1min":0.80600000000000005,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.554,"5min":1.575,"15min":1.575},"last":1.0820000000000001}]},{"osd":7,"last update":"Mon Apr 22 05:55:48 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.095,"15min":1.0209999999999999},"min":{"1min":0.86399999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.589,"5min":3.8159999999999998,"15min":3.8159999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.1200000000000001,"15min":1.0529999999999999},"min":{"1min":0.627,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.6299999999999999,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.3500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5579,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1042149376,"data_stored":1046890840,"data_compressed":29834343,"data_compressed_allocated":30949376,"data_compressed_original":36323328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":681115648,"data_stored":685252972,"data_compressed":26441385,"data_compressed_allocated":27336704,"data_compressed_original":31993856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417832960,"data_stored":419126404,"data_compressed":12754442,"data_compressed_allocated":13279232,"data_compressed_original":14893056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1040711680,"data_stored":1044304736,"data_compressed":24230251,"data_compressed_allocated":25128960,"data_compressed_original":29396992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1450180608,"data_stored":1463040808,"data_compressed":51243088,"data_compressed_allocated":53432320,"data_compressed_original":67096576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":863236096,"data_stored":869034460,"data_compressed":21037680,"data_compressed_allocated":21954560,"data_compressed_original":28217344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":318418944,"data_stored":318711896,"data_compressed":4811634,"data_compressed_allocated":5001216,"data_compressed_original":5484544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":982437888,"data_stored":987298932,"data_compressed":23922859,"data_compressed_allocated":24965120,"data_compressed_original":30486528,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T05:56:05.934 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-22T05:56:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24b149c640 1 Processor -- start 2024-04-22T05:56:06.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24b149c640 1 -- start start 2024-04-22T05:56:06.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24b149c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f24ac071a10 0x7f24ac13db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:56:06.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24b149c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f24ac1419e0 0x7f24ac141e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:56:06.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24b149c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ac0a8b00 con 0x7f24ac06e1a0 2024-04-22T05:56:06.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24b149c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ac0a7f30 con 0x7f24ac071a10 2024-04-22T05:56:06.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24b149c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac06e580 con 0x7f24ac1419e0 2024-04-22T05:56:06.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24aaffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f24ac06e1a0 0x7f24ac13d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42470/0 (socket says 172.21.15.40:42470) 2024-04-22T05:56:06.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24ab7fe640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f24ac1419e0 0x7f24ac141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:56:06.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24aaffd640 1 -- 172.21.15.40:0/230084670 learned_addr learned my addr 172.21.15.40:0/230084670 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-22T05:56:06.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24aa7fc640 1 --2- 172.21.15.40:0/230084670 >> v2:172.21.15.40:3300/0 conn(0x7f24ac071a10 0x7f24ac13db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:56:06.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24a9ffb640 1 -- 172.21.15.40:0/230084670 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 528636245 0 0) 0x7f24ac0a8b00 con 0x7f24ac06e1a0 2024-04-22T05:56:06.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24a9ffb640 1 -- 172.21.15.40:0/230084670 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2494003530 con 0x7f24ac06e1a0 2024-04-22T05:56:06.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.062+0000 7f24ab7fe640 1 -- 172.21.15.40:0/230084670 >> v2:172.21.15.40:3300/0 conn(0x7f24ac071a10 msgr2=0x7f24ac13db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T05:56:06.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24ab7fe640 1 --2- 172.21.15.40:0/230084670 >> v2:172.21.15.40:3300/0 conn(0x7f24ac071a10 0x7f24ac13db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:06.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24ab7fe640 1 -- 172.21.15.40:0/230084670 >> v1:172.21.15.40:6789/0 conn(0x7f24ac06e1a0 legacy=0x7f24ac13d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:56:06.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24ab7fe640 1 -- 172.21.15.40:0/230084670 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac13e180 con 0x7f24ac1419e0 2024-04-22T05:56:06.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24ab7fe640 1 --2- 172.21.15.40:0/230084670 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f24ac1419e0 0x7f24ac141e30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009d90 tx=0x7f24a401c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c7f0fe1c1f0202 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:56:06.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24a9ffb640 1 -- 172.21.15.40:0/230084670 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a40a5890 con 0x7f24ac1419e0 2024-04-22T05:56:06.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24a9ffb640 1 -- 172.21.15.40:0/230084670 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a40a59f0 con 0x7f24ac1419e0 2024-04-22T05:56:06.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24a9ffb640 1 -- 172.21.15.40:0/230084670 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a40a5d80 con 0x7f24ac1419e0 2024-04-22T05:56:06.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24b149c640 1 -- 172.21.15.40:0/230084670 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f24ac1419e0 msgr2=0x7f24ac141e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:56:06.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24b149c640 1 --2- 172.21.15.40:0/230084670 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f24ac1419e0 0x7f24ac141e30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009d90 tx=0x7f24a401c8c0 comp rx=0 tx=0).stop 2024-04-22T05:56:06.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24b149c640 1 -- 172.21.15.40:0/230084670 shutdown_connections 2024-04-22T05:56:06.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24b149c640 1 --2- 172.21.15.40:0/230084670 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f24ac1419e0 0x7f24ac141e30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:06.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24b149c640 1 --2- 172.21.15.40:0/230084670 >> v2:172.21.15.40:3300/0 conn(0x7f24ac071a10 0x7f24ac13db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:06.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24b149c640 1 -- 172.21.15.40:0/230084670 >> 172.21.15.40:0/230084670 conn(0x7f24ac0736e0 msgr2=0x7f24ac073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:56:06.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24b149c640 1 -- 172.21.15.40:0/230084670 shutdown_connections 2024-04-22T05:56:06.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.066+0000 7f24b149c640 1 -- 172.21.15.40:0/230084670 wait complete. 2024-04-22T05:56:06.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24b149c640 1 Processor -- start 2024-04-22T05:56:06.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24b149c640 1 -- start start 2024-04-22T05:56:06.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24b149c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24ac06e1a0 0x7f24ac134300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:56:06.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24b149c640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f24ac071a10 0x7f24ac14acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:56:06.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24b149c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24ac1419e0 0x7f24ac135b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24b149c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac135750 con 0x7f24ac06e1a0 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24b149c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac070580 con 0x7f24ac1419e0 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24b149c640 1 -- --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac136060 con 0x7f24ac071a10 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24aaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24ac06e1a0 0x7f24ac134300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24aa7fc640 1 --2- >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f24ac071a10 0x7f24ac14acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24aaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24ac06e1a0 0x7f24ac134300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53994/0 (socket says 172.21.15.40:53994) 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24aaffd640 1 -- 172.21.15.40:0/1398945323 learned_addr learned my addr 172.21.15.40:0/1398945323 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24aa7fc640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f24ac071a10 0x7f24ac14acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3301/0 says I am v2:172.21.15.40:42390/0 (socket says 172.21.15.40:42390) 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24ab7fe640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24ac1419e0 0x7f24ac135b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24aa7fc640 1 -- 172.21.15.40:0/1398945323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24ac1419e0 msgr2=0x7f24ac135b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24aa7fc640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24ac1419e0 0x7f24ac135b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24aa7fc640 1 -- 172.21.15.40:0/1398945323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24ac06e1a0 msgr2=0x7f24ac134300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24aa7fc640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24ac06e1a0 0x7f24ac134300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24aa7fc640 1 -- 172.21.15.40:0/1398945323 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a4009a70 con 0x7f24ac071a10 2024-04-22T05:56:06.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24aaffd640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24ac06e1a0 0x7f24ac134300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T05:56:06.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24ab7fe640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24ac1419e0 0x7f24ac135b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T05:56:06.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24aa7fc640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f24ac071a10 0x7f24ac14acb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f24a000e8a0 tx=0x7f24a000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:56:06.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24937fe640 1 -- 172.21.15.40:0/1398945323 <== mon.2 v2:172.21.15.40:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a000c940 con 0x7f24ac071a10 2024-04-22T05:56:06.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24937fe640 1 -- 172.21.15.40:0/1398945323 <== mon.2 v2:172.21.15.40:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a000caa0 con 0x7f24ac071a10 2024-04-22T05:56:06.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24b149c640 1 -- 172.21.15.40:0/1398945323 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ac132690 con 0x7f24ac071a10 2024-04-22T05:56:06.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24937fe640 1 -- 172.21.15.40:0/1398945323 <== mon.2 v2:172.21.15.40:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a000ce10 con 0x7f24ac071a10 2024-04-22T05:56:06.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24b149c640 1 -- 172.21.15.40:0/1398945323 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ac06fe20 con 0x7f24ac071a10 2024-04-22T05:56:06.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24b149c640 1 -- 172.21.15.40:0/1398945323 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ac0a8540 con 0x7f24ac071a10 2024-04-22T05:56:06.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24937fe640 1 -- 172.21.15.40:0/1398945323 <== mon.2 v2:172.21.15.40:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f24a0013470 con 0x7f24ac071a10 2024-04-22T05:56:06.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24937fe640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f2498066690 0x7f2498068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T05:56:06.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.070+0000 7f24937fe640 1 -- 172.21.15.40:0/1398945323 <== mon.2 v2:172.21.15.40:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f24a00841a0 con 0x7f24ac071a10 2024-04-22T05:56:06.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.074+0000 7f24aaffd640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f2498066690 0x7f2498068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T05:56:06.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.074+0000 7f24aaffd640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f2498066690 0x7f2498068b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f249c0011f0 tx=0x7f249c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T05:56:06.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.078+0000 7f24937fe640 1 -- 172.21.15.40:0/1398945323 <== mon.2 v2:172.21.15.40:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175051 (secure 0 0 0) 0x7f24a0051690 con 0x7f24ac071a10 2024-04-22T05:56:06.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.238+0000 7f24b149c640 1 -- 172.21.15.40:0/1398945323 --> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f24ac04e9c0 con 0x7f24ac071a10 2024-04-22T05:56:06.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.242+0000 7f24937fe640 1 -- 172.21.15.40:0/1398945323 <== mon.2 v2:172.21.15.40:3301/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f24a0051690 con 0x7f24ac071a10 2024-04-22T05:56:06.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.246+0000 7f24937fe640 1 -- 172.21.15.40:0/1398945323 <== mon.2 v2:172.21.15.40:3301/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7f24a0054cb0 con 0x7f24ac071a10 2024-04-22T05:56:06.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.246+0000 7f24b149c640 1 -- 172.21.15.40:0/1398945323 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f2498066690 msgr2=0x7f2498068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:56:06.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.246+0000 7f24b149c640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f2498066690 0x7f2498068b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f249c0011f0 tx=0x7f249c009040 comp rx=0 tx=0).stop 2024-04-22T05:56:06.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.246+0000 7f24b149c640 1 -- 172.21.15.40:0/1398945323 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f24ac071a10 msgr2=0x7f24ac14acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T05:56:06.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.246+0000 7f24b149c640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f24ac071a10 0x7f24ac14acb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f24a000e8a0 tx=0x7f24a000ed60 comp rx=0 tx=0).stop 2024-04-22T05:56:06.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.246+0000 7f24b149c640 1 -- 172.21.15.40:0/1398945323 shutdown_connections 2024-04-22T05:56:06.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.246+0000 7f24b149c640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.62:6832/24860,v1:172.21.15.62:6833/24860] conn(0x7f2498066690 0x7f2498068b50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:06.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.246+0000 7f24b149c640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24ac1419e0 0x7f24ac135b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:06.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.246+0000 7f24b149c640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.40:3301/0,v1:172.21.15.40:6790/0] conn(0x7f24ac071a10 0x7f24ac14acb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:06.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.246+0000 7f24b149c640 1 --2- 172.21.15.40:0/1398945323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24ac06e1a0 0x7f24ac134300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T05:56:06.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.246+0000 7f24b149c640 1 -- 172.21.15.40:0/1398945323 >> 172.21.15.40:0/1398945323 conn(0x7f24ac0736e0 msgr2=0x7f24ac132ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T05:56:06.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.250+0000 7f24b149c640 1 -- 172.21.15.40:0/1398945323 shutdown_connections 2024-04-22T05:56:06.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-22T05:56:06.250+0000 7f24b149c640 1 -- 172.21.15.40:0/1398945323 wait complete. 2024-04-22T05:56:06.267 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-22T05:56:06.268 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-22T05:56:06.268 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-22T05:56:06.268 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T05:56:10.450 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-04-22T05:56:12.370 INFO:tasks.ceph.osd.0:Stopped 2024-04-22T05:56:12.371 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-22T05:56:12.371 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T05:56:12.465 INFO:tasks.ceph.osd.1:Stopped 2024-04-22T05:56:12.465 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-22T05:56:12.465 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T05:56:16.458 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-22T05:56:16.458 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-04-22T05:56:18.567 INFO:tasks.ceph.osd.2:Stopped 2024-04-22T05:56:18.568 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-22T05:56:18.568 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T05:56:18.665 INFO:tasks.ceph.osd.3:Stopped 2024-04-22T05:56:18.666 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-22T05:56:18.666 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T05:56:22.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-04-22T05:56:22.265 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-22T05:56:24.768 INFO:tasks.ceph.osd.4:Stopped 2024-04-22T05:56:24.768 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-22T05:56:24.768 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T05:56:24.857 INFO:tasks.ceph.osd.5:Stopped 2024-04-22T05:56:24.857 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-22T05:56:24.857 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T05:56:27.870 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-22T05:56:27.871 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-04-22T05:56:30.959 INFO:tasks.ceph.osd.6:Stopped 2024-04-22T05:56:30.960 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-22T05:56:30.960 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T05:56:31.034 INFO:tasks.ceph.osd.7:Stopped 2024-04-22T05:56:31.034 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-22T05:56:31.034 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-22T05:56:31.034 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T05:56:31.061 INFO:tasks.ceph.mgr.y:Stopped 2024-04-22T05:56:31.061 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-22T05:56:31.061 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T05:56:31.091 INFO:tasks.ceph.mgr.x:Stopped 2024-04-22T05:56:31.091 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-22T05:56:31.092 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-22T05:56:31.092 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T05:56:31.123 INFO:tasks.ceph.mon.a:Stopped 2024-04-22T05:56:31.123 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-22T05:56:31.123 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T05:56:31.186 INFO:tasks.ceph.mon.c:Stopped 2024-04-22T05:56:31.186 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-22T05:56:31.186 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T05:56:31.211 INFO:tasks.ceph.mon.b:Stopped 2024-04-22T05:56:31.211 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-22T05:56:31.211 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | head -n 1 2024-04-22T05:56:31.245 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-22T05:56:31.245 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-22T05:56:31.530 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-22T05:56:31.530 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-22T05:56:31.743 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-22T05:56:31.743 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-22T05:56:31.911 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-22T05:56:31.911 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-22T05:56:32.106 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-22T05:56:32.107 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-22T05:56:32.331 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-22T05:56:32.331 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-22T05:56:32.493 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-22T05:56:32.493 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-22T05:56:32.652 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-22T05:56:32.652 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-22T05:56:32.803 INFO:tasks.ceph:Archiving mon data... 2024-04-22T05:56:32.804 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776/data/mon.a.tgz 2024-04-22T05:56:32.804 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-04-22T05:56:32.810 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.mqJIcpqwQs 2024-04-22T05:56:32.810 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.mqJIcpqwQs 2024-04-22T05:56:32.872 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-22T05:56:32.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.mqJIcpqwQs 2024-04-22T05:56:33.000 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.mqJIcpqwQs is 316KB 2024-04-22T05:56:33.064 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.mqJIcpqwQs 2024-04-22T05:56:33.070 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776/data/mon.c.tgz 2024-04-22T05:56:33.070 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-04-22T05:56:33.123 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.5v5Fb38xVK 2024-04-22T05:56:33.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.5v5Fb38xVK 2024-04-22T05:56:33.246 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.5v5Fb38xVK 2024-04-22T05:56:33.311 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.5v5Fb38xVK is 317KB 2024-04-22T05:56:33.368 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.5v5Fb38xVK 2024-04-22T05:56:33.373 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776/data/mon.b.tgz 2024-04-22T05:56:33.373 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-04-22T05:56:33.380 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.KZHmNALSYx 2024-04-22T05:56:33.380 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.KZHmNALSYx 2024-04-22T05:56:33.496 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.KZHmNALSYx 2024-04-22T05:56:33.563 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.KZHmNALSYx is 317KB 2024-04-22T05:56:33.620 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.KZHmNALSYx 2024-04-22T05:56:33.626 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-22T05:56:33.626 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-22T05:56:33.629 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-22T05:56:33.683 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-04-22T05:56:33.683 DEBUG:teuthology.orchestra.run.smithi040:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-22T05:56:33.685 DEBUG:teuthology.orchestra.run.smithi062:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-22T05:56:33.717 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-22T05:56:33.748 INFO:teuthology.orchestra.run.smithi062.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-22T05:56:33.752 INFO:tasks.ceph:Archiving crash dumps... 2024-04-22T05:56:33.754 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776/remote/smithi040/crash 2024-04-22T05:56:33.755 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-22T05:56:33.776 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776/remote/smithi062/crash 2024-04-22T05:56:33.777 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-22T05:56:33.806 INFO:tasks.ceph:Compressing logs... 2024-04-22T05:56:33.806 DEBUG:teuthology.orchestra.run.smithi040:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T05:56:33.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T05:57:39.565 INFO:tasks.ceph:Archiving logs... 2024-04-22T05:57:39.566 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776/remote/smithi040/log 2024-04-22T05:57:39.570 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T05:57:46.100 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776/remote/smithi062/log 2024-04-22T05:57:46.101 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T05:57:52.915 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-22T05:57:52.928 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-22T05:57:52.928 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T05:57:52.931 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T05:57:53.000 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-22T05:57:53.009 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-22T05:57:53.009 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T05:57:53.011 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T05:57:53.022 INFO:teuthology.orchestra.run.smithi040.stdout: remote refid st t when poll reach delay offset jitter 2024-04-22T05:57:53.022 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================== 2024-04-22T05:57:53.022 INFO:teuthology.orchestra.run.smithi040.stdout:*hv01.front.sepi 44.4.53.2 2 u 11 64 377 0.090 -1.921 0.706 2024-04-22T05:57:53.022 INFO:teuthology.orchestra.run.smithi040.stdout:+hv02.front.sepi 63.231.80.2 3 u 8 64 377 0.068 +0.257 0.548 2024-04-22T05:57:53.022 INFO:teuthology.orchestra.run.smithi040.stdout:+hv03.front.sepi 74.6.168.72 3 u 9 64 377 0.086 -0.920 0.326 2024-04-22T05:57:53.022 INFO:teuthology.orchestra.run.smithi040.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 +0.000 0.000 2024-04-22T05:57:53.024 INFO:teuthology.orchestra.run.smithi062.stdout: remote refid st t when poll reach delay offset jitter 2024-04-22T05:57:53.025 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================== 2024-04-22T05:57:53.025 INFO:teuthology.orchestra.run.smithi062.stdout:*hv01.front.sepi 44.4.53.2 2 u 10 64 377 0.070 -2.543 0.803 2024-04-22T05:57:53.025 INFO:teuthology.orchestra.run.smithi062.stdout:+hv02.front.sepi 63.231.80.2 3 u 15 64 377 0.090 -0.231 0.794 2024-04-22T05:57:53.025 INFO:teuthology.orchestra.run.smithi062.stdout:+hv03.front.sepi 74.6.168.72 3 u 9 64 377 0.083 -2.052 0.849 2024-04-22T05:57:53.025 INFO:teuthology.orchestra.run.smithi062.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 +0.000 0.000 2024-04-22T05:57:53.026 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-22T05:57:53.034 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-22T05:57:53.035 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-22T05:57:53.043 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-22T05:57:53.053 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-22T05:57:53.063 INFO:teuthology.task.internal:Duration was 1033.541776 seconds 2024-04-22T05:57:53.063 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-22T05:57:53.073 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-22T05:57:53.073 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T05:57:53.076 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T05:57:53.114 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-22T05:57:53.114 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2024-04-22T05:57:53.114 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T05:57:53.171 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-04-22T05:57:53.172 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T05:57:53.182 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-22T05:57:53.182 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T05:57:53.219 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T05:57:53.250 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-22T05:57:53.250 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T05:57:53.287 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T05:57:53.348 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-22T05:57:53.359 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-22T05:57:53.359 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T05:57:53.372 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T05:57:53.383 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-22T05:57:53.393 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T05:57:53.416 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T05:57:53.425 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2024-04-22T05:57:53.435 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-04-22T05:57:53.457 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T05:57:53.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T05:57:53.482 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T05:57:53.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T05:57:53.506 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-22T05:57:53.516 INFO:teuthology.task.internal:Transferring archived files... 2024-04-22T05:57:53.517 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776/remote/smithi040 2024-04-22T05:57:53.517 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T05:57:53.547 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-21_22:00:02-rados-reef-distro-default-smithi/7667776/remote/smithi062 2024-04-22T05:57:53.547 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T05:57:53.571 INFO:teuthology.task.internal:Removing archive directory... 2024-04-22T05:57:53.571 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T05:57:53.587 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T05:57:53.613 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-22T05:57:53.624 INFO:teuthology.task.internal:Not uploading archives. 2024-04-22T05:57:53.624 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-22T05:57:53.652 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-22T05:57:53.652 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T05:57:53.655 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T05:57:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: 658643 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 05:57 /home/ubuntu/cephtest 2024-04-22T05:57:53.662 INFO:teuthology.orchestra.run.smithi062.stdout: 658640 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 05:57 /home/ubuntu/cephtest 2024-04-22T05:57:53.665 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-22T05:57:53.675 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-22T05:57:53.740 INFO:teuthology.run:Summary data: description: rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} duration: 1033.541776418686 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-22T05:57:53.741 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T05:57:53.877 INFO:teuthology.run:pass