2024-09-18T07:03:59.410 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-18T07:03:59.428 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T07:03:59.519 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi/7910278 branch: main description: rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} tasks/libcephsqlite} email: null first_in_suite: false job_id: '7910278' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: debug cephsqlite: 20 debug client: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 1 ms bind msgr2: false ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - POOL_APP_NOT_ENABLED - do not have an application enabled mon_bind_msgr2: false sha1: 82e3d5962b49b7d08ddbe09b754f372dd36156d5 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 82e3d5962b49b7d08ddbe09b754f372dd36156d5 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 0325fd3c11a225c0fd9eb38fab88aad1370ad779 owner: scheduled_nmordech@teuthology priority: 50 repo: https://github.com/ceph/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 4523 sha1: 82e3d5962b49b7d08ddbe09b754f372dd36156d5 sleep_before_teardown: 0 suite: rados:basic suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 0325fd3c11a225c0fd9eb38fab88aad1370ad779 targets: smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQAz9ntCMJqZQnLilFX1KjpYnTB2hDn3MLfk3XGbitpB1/BNoar2fSwEkkaw4D//uMkMEtKtBWGupn3wyPJB2w= smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGqDrmQeae9RbqQO8dH0iE6uRpgU54gpeMteFMngd6EDPAt3MURb9IohvkN0e2s8ELnsb258r8liooylziVa++Q= tasks: - install: extra_system_packages: deb: - sqlite3 rpm: - sqlite-devel - ceph: null - exec: client.0: - ceph osd pool create cephsqlite - ceph auth get-or-create client.libcephsqlite mon 'profile simple-rados-client-with-blocklist' osd 'allow rwx pool=cephsqlite' >> /etc/ceph/ceph.keyring - exec: client.0: - ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr - workunit: clients: client.0: - rados/test_libcephsqlite.sh cephsqlite env: CEPH_ARGS: --id libcephsqlite --no-log-to-stderr teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_06:58:32 tube: smithi user: nmordech verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-18T07:03:59.519 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa; will attempt to use it 2024-09-18T07:03:59.547 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa/tasks 2024-09-18T07:03:59.548 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-18T07:03:59.549 INFO:teuthology.task.internal:Checking packages... 2024-09-18T07:03:59.572 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '82e3d5962b49b7d08ddbe09b754f372dd36156d5' 2024-09-18T07:03:59.572 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-18T07:03:59.572 INFO:teuthology.packaging:ref: None 2024-09-18T07:03:59.572 INFO:teuthology.packaging:tag: None 2024-09-18T07:03:59.573 INFO:teuthology.packaging:branch: main 2024-09-18T07:03:59.573 INFO:teuthology.packaging:sha1: 82e3d5962b49b7d08ddbe09b754f372dd36156d5 2024-09-18T07:03:59.573 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-09-18T07:03:59.781 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4987.g5ecc7400 2024-09-18T07:03:59.783 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-18T07:03:59.789 INFO:teuthology.task.internal:no buildpackages task found 2024-09-18T07:03:59.789 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-18T07:03:59.802 INFO:teuthology.task.internal:Saving configuration 2024-09-18T07:03:59.816 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-18T07:03:59.822 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-18T07:03:59.848 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi/7910278', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 06:58:55.045273', 'locked_by': 'scheduled_nmordech@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQAz9ntCMJqZQnLilFX1KjpYnTB2hDn3MLfk3XGbitpB1/BNoar2fSwEkkaw4D//uMkMEtKtBWGupn3wyPJB2w='} 2024-09-18T07:03:59.870 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi/7910278', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 06:58:55.047206', 'locked_by': 'scheduled_nmordech@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGqDrmQeae9RbqQO8dH0iE6uRpgU54gpeMteFMngd6EDPAt3MURb9IohvkN0e2s8ELnsb258r8liooylziVa++Q='} 2024-09-18T07:03:59.870 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-18T07:03:59.876 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-18T07:03:59.876 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-18T07:03:59.876 INFO:teuthology.run_tasks:Running task console_log... 2024-09-18T07:03:59.945 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa2a8bbbac0>, signals=[15]) 2024-09-18T07:03:59.945 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-18T07:03:59.951 INFO:teuthology.task.internal:Opening connections... 2024-09-18T07:03:59.951 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2024-09-18T07:03:59.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T07:04:00.034 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2024-09-18T07:04:00.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T07:04:00.112 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-18T07:04:00.118 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2024-09-18T07:04:00.136 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2024-09-18T07:04:00.137 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-09-18T07:04:00.195 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-09-18T07:04:00.195 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2024-09-18T07:04:00.196 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-09-18T07:04:00.196 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-09-18T07:04:00.196 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2024-09-18T07:04:00.196 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2024-09-18T07:04:00.196 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-18T07:04:00.197 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-09-18T07:04:00.197 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2024-09-18T07:04:00.197 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-18T07:04:00.197 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-09-18T07:04:00.197 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-18T07:04:00.197 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-18T07:04:00.198 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-18T07:04:00.199 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-09-18T07:04:00.224 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2024-09-18T07:04:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2024-09-18T07:04:00.243 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2024-09-18T07:04:00.300 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2024-09-18T07:04:00.300 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2024-09-18T07:04:00.300 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2024-09-18T07:04:00.300 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2024-09-18T07:04:00.300 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2024-09-18T07:04:00.300 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2024-09-18T07:04:00.300 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-18T07:04:00.301 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2024-09-18T07:04:00.301 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2024-09-18T07:04:00.301 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-18T07:04:00.301 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2024-09-18T07:04:00.301 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-18T07:04:00.301 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-18T07:04:00.301 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-18T07:04:00.301 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2024-09-18T07:04:00.323 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-18T07:04:00.332 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-18T07:04:00.342 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-18T07:04:00.342 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2024-09-18T07:04:00.344 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2024-09-18T07:04:00.361 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-18T07:04:00.369 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-18T07:04:00.369 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2024-09-18T07:04:00.402 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2024-09-18T07:04:00.440 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-18T07:04:00.497 INFO:teuthology.run_tasks:Running task kernel... 2024-09-18T07:04:00.511 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-18T07:04:00.512 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-18T07:04:00.512 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T07:04:00.512 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T07:04:00.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T07:04:00.529 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-18T07:04:00.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T07:04:00.530 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2024-09-18T07:04:00.587 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-503.el9.x86_64 2024-09-18T07:04:00.588 INFO:teuthology.task.kernel:Running kernel on smithi070: 5.14.0-503.el9.x86_64 2024-09-18T07:04:00.588 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum install -y kernel 2024-09-18T07:04:00.589 INFO:teuthology.orchestra.run.smithi172.stdout:5.14.0-503.el9.x86_64 2024-09-18T07:04:00.589 INFO:teuthology.task.kernel:Running kernel on smithi172: 5.14.0-503.el9.x86_64 2024-09-18T07:04:00.589 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum install -y kernel 2024-09-18T07:04:03.184 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.3 MB 00:00 2024-09-18T07:04:04.918 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-18T07:04:05.152 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.3 MB 00:02 2024-09-18T07:04:09.127 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-18T07:04:09.852 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 20 MB 00:03 2024-09-18T07:04:10.519 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 99 kB/s | 19 kB 00:00 2024-09-18T07:04:12.607 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 11 MB/s | 23 MB 00:01 2024-09-18T07:04:14.336 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.5 MB 00:00 2024-09-18T07:04:15.886 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 19 kB 00:00 2024-09-18T07:04:17.515 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-18T07:04:18.054 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-18T07:04:20.019 INFO:teuthology.orchestra.run.smithi172.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-18T07:04:20.019 INFO:teuthology.orchestra.run.smithi172.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-18T07:04:20.112 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-18T07:04:20.117 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-18T07:04:20.117 INFO:teuthology.orchestra.run.smithi172.stdout: Package Architecture Version Repository Size 2024-09-18T07:04:20.117 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-18T07:04:20.117 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-18T07:04:20.117 INFO:teuthology.orchestra.run.smithi172.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-18T07:04:20.117 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-18T07:04:20.118 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-18T07:04:20.118 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-18T07:04:20.118 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-18T07:04:20.118 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:04:20.118 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-18T07:04:20.118 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-18T07:04:20.118 INFO:teuthology.orchestra.run.smithi172.stdout:Install 4 Packages 2024-09-18T07:04:20.118 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:04:20.119 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 79 M 2024-09-18T07:04:20.119 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 126 M 2024-09-18T07:04:20.119 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-18T07:04:20.272 INFO:teuthology.orchestra.run.smithi172.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 274 kB/s | 36 kB 00:00 2024-09-18T07:04:21.397 INFO:teuthology.orchestra.run.smithi172.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-09-18T07:04:21.906 INFO:teuthology.orchestra.run.smithi172.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 17 MB/s | 29 MB 00:01 2024-09-18T07:04:22.314 INFO:teuthology.orchestra.run.smithi172.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-18T07:04:22.314 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-18T07:04:22.315 INFO:teuthology.orchestra.run.smithi172.stdout:Total 36 MB/s | 79 MB 00:02 2024-09-18T07:04:22.854 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-18T07:04:22.997 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-18T07:04:23.356 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-18T07:04:23.357 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-18T07:04:24.355 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-18T07:04:24.356 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-18T07:04:24.785 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-18T07:04:24.785 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-18T07:04:24.885 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-18T07:04:24.900 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-18T07:04:24.901 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-18T07:04:24.901 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-18T07:04:24.901 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-18T07:04:24.901 INFO:teuthology.orchestra.run.smithi070.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-18T07:04:24.901 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-18T07:04:24.901 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-18T07:04:24.901 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-18T07:04:24.901 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-18T07:04:24.901 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:04:24.901 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-18T07:04:24.901 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-18T07:04:24.901 INFO:teuthology.orchestra.run.smithi070.stdout:Install 4 Packages 2024-09-18T07:04:24.901 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:04:24.902 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 79 M 2024-09-18T07:04:24.902 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 126 M 2024-09-18T07:04:24.902 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-18T07:04:25.005 INFO:teuthology.orchestra.run.smithi070.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 433 kB/s | 36 kB 00:00 2024-09-18T07:04:26.790 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-18T07:04:27.419 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-18T07:04:27.507 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T07:04:28.772 INFO:teuthology.orchestra.run.smithi070.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 4.1 MB/s | 16 MB 00:03 2024-09-18T07:04:30.283 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T07:04:30.419 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T07:04:32.246 INFO:teuthology.orchestra.run.smithi070.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 3.9 MB/s | 29 MB 00:07 2024-09-18T07:04:33.838 INFO:teuthology.orchestra.run.smithi070.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 3.9 MB/s | 35 MB 00:08 2024-09-18T07:04:33.838 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-18T07:04:33.838 INFO:teuthology.orchestra.run.smithi070.stdout:Total 8.8 MB/s | 79 MB 00:08 2024-09-18T07:04:34.527 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-18T07:04:34.895 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-18T07:04:34.895 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-18T07:04:35.231 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T07:04:35.278 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T07:04:35.869 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-18T07:04:35.870 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-18T07:04:38.303 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-18T07:04:38.943 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-18T07:04:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T07:04:40.156 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T07:04:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T07:04:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T07:04:46.753 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T07:04:47.041 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T07:04:51.870 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T07:05:18.308 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T07:05:18.351 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-18T07:05:18.696 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T07:05:18.697 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-18T07:05:18.697 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T07:05:18.697 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T07:05:19.526 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T07:05:19.526 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:05:19.526 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-18T07:05:19.526 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-18T07:05:19.526 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-18T07:05:19.526 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-18T07:05:19.526 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-18T07:05:19.526 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:05:19.526 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-18T07:05:19.875 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T07:05:19.961 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T07:05:19.961 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T07:05:19.961 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-18T07:05:19.962 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T07:05:19.962 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T07:05:19.962 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-18T07:05:19.962 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi172.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T07:05:19.962 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T07:05:20.021 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T07:05:20.021 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T07:05:20.021 DEBUG:teuthology.orchestra.run.smithi172:> sudo rpm -qi grub2-tools 2024-09-18T07:05:20.061 INFO:teuthology.orchestra.run.smithi172.stdout:Name : grub2-tools 2024-09-18T07:05:20.062 INFO:teuthology.orchestra.run.smithi172.stdout:Epoch : 1 2024-09-18T07:05:20.062 INFO:teuthology.orchestra.run.smithi172.stdout:Version : 2.06 2024-09-18T07:05:20.062 INFO:teuthology.orchestra.run.smithi172.stdout:Release : 82.el9 2024-09-18T07:05:20.062 INFO:teuthology.orchestra.run.smithi172.stdout:Architecture: x86_64 2024-09-18T07:05:20.062 INFO:teuthology.orchestra.run.smithi172.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-18T07:05:20.062 INFO:teuthology.orchestra.run.smithi172.stdout:Group : Unspecified 2024-09-18T07:05:20.062 INFO:teuthology.orchestra.run.smithi172.stdout:Size : 8274670 2024-09-18T07:05:20.062 INFO:teuthology.orchestra.run.smithi172.stdout:License : GPLv3+ 2024-09-18T07:05:20.062 INFO:teuthology.orchestra.run.smithi172.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-18T07:05:20.062 INFO:teuthology.orchestra.run.smithi172.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-18T07:05:20.062 INFO:teuthology.orchestra.run.smithi172.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-18T07:05:20.062 INFO:teuthology.orchestra.run.smithi172.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-18T07:05:20.062 INFO:teuthology.orchestra.run.smithi172.stdout:Packager : builder@centos.org 2024-09-18T07:05:20.063 INFO:teuthology.orchestra.run.smithi172.stdout:Vendor : CentOS 2024-09-18T07:05:20.063 INFO:teuthology.orchestra.run.smithi172.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-18T07:05:20.063 INFO:teuthology.orchestra.run.smithi172.stdout:Summary : Support tools for GRUB. 2024-09-18T07:05:20.063 INFO:teuthology.orchestra.run.smithi172.stdout:Description : 2024-09-18T07:05:20.063 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:05:20.063 INFO:teuthology.orchestra.run.smithi172.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-18T07:05:20.063 INFO:teuthology.orchestra.run.smithi172.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-18T07:05:20.063 INFO:teuthology.orchestra.run.smithi172.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-18T07:05:20.063 INFO:teuthology.orchestra.run.smithi172.stdout:hardware devices. 2024-09-18T07:05:20.063 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:05:20.063 INFO:teuthology.orchestra.run.smithi172.stdout:This subpackage provides tools for support of all platforms. 2024-09-18T07:05:20.064 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-18T07:05:20.064 INFO:teuthology.task.kernel:Updating grub on smithi172 to boot 5.14.0-508.el9.x86_64 2024-09-18T07:05:20.065 DEBUG:teuthology.orchestra.run.smithi172:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-18T07:05:20.732 INFO:teuthology.orchestra.run.smithi172.stderr:Generating grub configuration file ... 2024-09-18T07:05:22.088 INFO:teuthology.orchestra.run.smithi172.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-18T07:05:22.112 INFO:teuthology.orchestra.run.smithi172.stderr:done 2024-09-18T07:05:22.114 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2024-09-18T07:05:22.130 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.MdYHZY7Hq8 2024-09-18T07:05:22.130 DEBUG:teuthology.orchestra.run.smithi172:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MdYHZY7Hq8 2024-09-18T07:05:22.197 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.MdYHZY7Hq8 2024-09-18T07:05:22.336 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.MdYHZY7Hq8 is 6KB 2024-09-18T07:05:22.385 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.MdYHZY7Hq8 2024-09-18T07:05:22.399 DEBUG:teuthology.orchestra.run.smithi172:> sudo /bin/ls /boot/loader/entries || true 2024-09-18T07:05:22.462 INFO:teuthology.orchestra.run.smithi172.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-18T07:05:22.462 INFO:teuthology.orchestra.run.smithi172.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-18T07:05:22.462 INFO:teuthology.orchestra.run.smithi172.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-18T07:05:22.462 INFO:teuthology.orchestra.run.smithi172.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-18T07:05:22.465 DEBUG:teuthology.orchestra.run.smithi172:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-18T07:05:22.619 DEBUG:teuthology.orchestra.run.smithi172:> sudo shutdown -r now 2024-09-18T07:05:30.080 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T07:05:30.122 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-18T07:05:30.471 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T07:05:30.471 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-18T07:05:30.471 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T07:05:30.471 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T07:05:31.307 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T07:05:31.308 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:05:31.308 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-18T07:05:31.308 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-18T07:05:31.308 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-18T07:05:31.308 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-18T07:05:31.308 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-18T07:05:31.308 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:05:31.308 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-18T07:05:31.655 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T07:05:31.745 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T07:05:31.745 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T07:05:31.745 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-18T07:05:31.745 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T07:05:31.746 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T07:05:31.746 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-18T07:05:31.746 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T07:05:31.746 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T07:05:31.804 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T07:05:31.804 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T07:05:31.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2024-09-18T07:05:31.838 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2024-09-18T07:05:31.838 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2024-09-18T07:05:31.838 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.06 2024-09-18T07:05:31.838 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 82.el9 2024-09-18T07:05:31.838 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2024-09-18T07:05:31.838 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-18T07:05:31.838 INFO:teuthology.orchestra.run.smithi070.stdout:Group : Unspecified 2024-09-18T07:05:31.839 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 8274670 2024-09-18T07:05:31.839 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2024-09-18T07:05:31.839 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-18T07:05:31.839 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-18T07:05:31.839 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-18T07:05:31.839 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-18T07:05:31.839 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : builder@centos.org 2024-09-18T07:05:31.839 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : CentOS 2024-09-18T07:05:31.839 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-18T07:05:31.839 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2024-09-18T07:05:31.839 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2024-09-18T07:05:31.839 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:05:31.839 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-18T07:05:31.840 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-18T07:05:31.840 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-18T07:05:31.840 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2024-09-18T07:05:31.840 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:05:31.840 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2024-09-18T07:05:31.841 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-18T07:05:31.841 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 5.14.0-508.el9.x86_64 2024-09-18T07:05:31.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-18T07:05:32.517 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2024-09-18T07:05:33.835 INFO:teuthology.orchestra.run.smithi070.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-18T07:05:33.873 INFO:teuthology.orchestra.run.smithi070.stderr:done 2024-09-18T07:05:33.875 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-18T07:05:33.890 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.805BoaxiAV 2024-09-18T07:05:33.890 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.805BoaxiAV 2024-09-18T07:05:33.957 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.805BoaxiAV 2024-09-18T07:05:34.101 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.805BoaxiAV is 6KB 2024-09-18T07:05:34.149 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.805BoaxiAV 2024-09-18T07:05:34.163 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2024-09-18T07:05:34.231 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-18T07:05:34.232 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-18T07:05:34.232 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-18T07:05:34.232 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-18T07:05:34.233 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-18T07:05:34.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2024-09-18T07:05:52.640 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-18T07:05:52.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-18T07:05:52.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T07:06:04.401 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-18T07:06:04.402 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-18T07:06:04.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T07:06:11.133 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-09-18T07:06:20.141 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-18T07:06:20.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T07:06:22.909 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-09-18T07:06:23.197 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-09-18T07:06:31.917 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-18T07:06:31.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T07:06:35.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-18T07:06:35.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T07:06:53.629 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-09-18T07:07:08.645 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-18T07:07:08.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T07:07:09.049 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-09-18T07:07:09.425 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-18T07:07:09.425 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-18T07:07:09.426 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2024-09-18T07:07:09.441 INFO:teuthology.orchestra.run.smithi172.stdout:5.14.0-508.el9.x86_64 2024-09-18T07:07:09.441 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-18T07:07:09.441 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T07:07:09.441 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-18T07:07:10.442 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-18T07:07:10.442 DEBUG:teuthology.orchestra.run.smithi172:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T07:07:10.542 INFO:teuthology.orchestra.run.smithi172.stdout:ttyS1 2024-09-18T07:07:10.573 DEBUG:teuthology.parallel:result is None 2024-09-18T07:07:31.920 DEBUG:teuthology.orchestra.remote:timed out 2024-09-18T07:07:43.922 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-18T07:07:43.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T07:07:44.316 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-18T07:07:44.648 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-18T07:07:44.648 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-18T07:07:44.648 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-18T07:07:44.664 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-508.el9.x86_64 2024-09-18T07:07:44.664 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-18T07:07:44.664 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T07:07:44.664 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-18T07:07:45.665 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-18T07:07:45.666 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T07:07:45.745 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2024-09-18T07:07:45.762 DEBUG:teuthology.parallel:result is None 2024-09-18T07:07:45.763 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-18T07:07:45.772 INFO:teuthology.task.internal:Creating test directory... 2024-09-18T07:07:45.772 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T07:07:45.775 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T07:07:45.795 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-18T07:07:45.807 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-18T07:07:45.822 INFO:teuthology.task.internal:Creating archive directory... 2024-09-18T07:07:45.822 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T07:07:45.833 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T07:07:45.926 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-18T07:07:45.936 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-18T07:07:45.936 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T07:07:45.940 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T07:07:45.976 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T07:07:45.976 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T07:07:45.987 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T07:07:45.987 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T07:07:45.990 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-18T07:07:46.000 INFO:teuthology.task.internal:Configuring sudo... 2024-09-18T07:07:46.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T07:07:46.032 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T07:07:46.061 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-18T07:07:46.075 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-18T07:07:46.075 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T07:07:46.100 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T07:07:46.118 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T07:07:46.202 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T07:07:46.245 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T07:07:46.329 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T07:07:46.370 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:07:46.370 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T07:07:46.435 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T07:07:46.481 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T07:07:46.540 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T07:07:46.623 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T07:07:46.667 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:07:46.667 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T07:07:46.733 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2024-09-18T07:07:46.735 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2024-09-18T07:07:46.777 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T07:07:46.820 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T07:07:47.289 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-18T07:07:47.297 INFO:teuthology.task.internal:Starting timer... 2024-09-18T07:07:47.297 INFO:teuthology.run_tasks:Running task pcp... 2024-09-18T07:07:47.309 INFO:teuthology.run_tasks:Running task selinux... 2024-09-18T07:07:47.320 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-18T07:07:47.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2024-09-18T07:07:47.370 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2024-09-18T07:07:47.372 DEBUG:teuthology.orchestra.run.smithi172:> sudo service auditd rotate 2024-09-18T07:07:47.444 INFO:teuthology.orchestra.run.smithi172.stdout:Rotating logs: 2024-09-18T07:07:47.446 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-18T07:07:47.446 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2024-09-18T07:07:47.473 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2024-09-18T07:07:47.474 DEBUG:teuthology.orchestra.run.smithi172:> /usr/sbin/getenforce 2024-09-18T07:07:47.502 INFO:teuthology.orchestra.run.smithi172.stdout:Permissive 2024-09-18T07:07:47.502 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi070.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi172.front.sepia.ceph.com': 'permissive'} 2024-09-18T07:07:47.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T07:07:47.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T07:07:47.541 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T07:07:47.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T07:07:47.572 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-18T07:07:47.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2024-09-18T07:07:47.620 DEBUG:teuthology.orchestra.run.smithi172:> sudo /usr/sbin/setenforce permissive 2024-09-18T07:07:47.656 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-18T07:07:47.666 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-18T07:07:47.669 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-18T07:07:47.779 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-18T07:07:47.797 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-18T07:07:47.799 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi070.front.sepia.ceph.com,smithi172.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-18T07:11:37.945 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi070.front.sepia.ceph.com'), Remote(name='ubuntu@smithi172.front.sepia.ceph.com')] 2024-09-18T07:11:37.947 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-18T07:11:37.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T07:11:38.028 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-18T07:11:38.104 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-18T07:11:38.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-18T07:11:38.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T07:11:38.182 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-09-18T07:11:38.267 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-18T07:11:38.267 INFO:teuthology.run_tasks:Running task clock... 2024-09-18T07:11:38.278 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-18T07:11:38.278 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T07:11:38.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T07:11:38.281 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T07:11:38.282 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T07:11:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-18T07:11:38.321 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-18T07:11:38.340 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-18T07:11:38.346 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2024-09-18T07:11:38.356 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-18T07:11:38.357 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2024-09-18T07:11:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-18T07:11:38.383 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: ntpd: command not found 2024-09-18T07:11:38.385 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-18T07:11:38.394 INFO:teuthology.orchestra.run.smithi172.stdout:506 Cannot talk to daemon 2024-09-18T07:11:38.410 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-18T07:11:38.424 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-18T07:11:38.440 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-18T07:11:38.444 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T07:11:38.444 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-18T07:11:38.444 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T07:11:38.444 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T07:11:38.444 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T07:11:38.444 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T07:11:38.490 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-09-18T07:11:38.493 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T07:11:38.493 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-09-18T07:11:38.494 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T07:11:38.494 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T07:11:38.494 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T07:11:38.494 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T07:11:38.495 INFO:teuthology.run_tasks:Running task install... 2024-09-18T07:11:38.507 DEBUG:teuthology.task.install:project ceph 2024-09-18T07:11:38.507 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '82e3d5962b49b7d08ddbe09b754f372dd36156d5'}} 2024-09-18T07:11:38.507 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': '82e3d5962b49b7d08ddbe09b754f372dd36156d5'} 2024-09-18T07:11:38.507 INFO:teuthology.task.install:Using flavor: default 2024-09-18T07:11:38.527 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-18T07:11:38.528 INFO:teuthology.task.install:extra packages: [] 2024-09-18T07:11:38.528 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '82e3d5962b49b7d08ddbe09b754f372dd36156d5', 'tag': None, 'wait_for_package': False} 2024-09-18T07:11:38.529 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=82e3d5962b49b7d08ddbe09b754f372dd36156d5 2024-09-18T07:11:38.531 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '82e3d5962b49b7d08ddbe09b754f372dd36156d5', 'tag': None, 'wait_for_package': False} 2024-09-18T07:11:38.531 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=82e3d5962b49b7d08ddbe09b754f372dd36156d5 2024-09-18T07:11:38.681 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/82e3d5962b49b7d08ddbe09b754f372dd36156d5/centos/9/flavors/default/ 2024-09-18T07:11:38.681 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4953.g82e3d596 2024-09-18T07:11:38.684 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/82e3d5962b49b7d08ddbe09b754f372dd36156d5/centos/9/flavors/default/ 2024-09-18T07:11:38.684 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4953.g82e3d596 2024-09-18T07:11:38.911 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/82e3d5962b49b7d08ddbe09b754f372dd36156d5/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/82e3d5962b49b7d08ddbe09b754f372dd36156d5/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/82e3d5962b49b7d08ddbe09b754f372dd36156d5/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-18T07:11:38.911 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:11:38.911 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-18T07:11:38.914 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/82e3d5962b49b7d08ddbe09b754f372dd36156d5/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/82e3d5962b49b7d08ddbe09b754f372dd36156d5/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/82e3d5962b49b7d08ddbe09b754f372dd36156d5/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-18T07:11:38.914 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:11:38.914 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-18T07:11:38.937 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, sqlite-devel, sqlite-devel, sqlite-devel on remote rpm x86_64 2024-09-18T07:11:38.938 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/82e3d5962b49b7d08ddbe09b754f372dd36156d5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-18T07:11:38.941 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, sqlite-devel, sqlite-devel, sqlite-devel, sqlite-devel on remote rpm x86_64 2024-09-18T07:11:38.941 DEBUG:teuthology.orchestra.run.smithi172:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/82e3d5962b49b7d08ddbe09b754f372dd36156d5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-18T07:11:39.004 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-18T07:11:39.008 DEBUG:teuthology.orchestra.run.smithi172:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-18T07:11:39.080 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-18T07:11:39.083 DEBUG:teuthology.orchestra.run.smithi172:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-18T07:11:39.144 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2024-09-18T07:11:39.146 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2024-09-18T07:11:39.147 INFO:teuthology.orchestra.run.smithi172.stdout:check_obsoletes = 1 2024-09-18T07:11:39.149 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum clean all 2024-09-18T07:11:39.377 INFO:teuthology.orchestra.run.smithi070.stdout:45 files removed 2024-09-18T07:11:39.381 INFO:teuthology.orchestra.run.smithi172.stdout:45 files removed 2024-09-18T07:11:39.404 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd sqlite-devel sqlite-devel sqlite-devel sqlite-devel 2024-09-18T07:11:39.408 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd sqlite-devel sqlite-devel sqlite-devel sqlite-devel 2024-09-18T07:11:40.205 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 162 kB/s | 84 kB 00:00 2024-09-18T07:11:40.242 INFO:teuthology.orchestra.run.smithi172.stdout:ceph packages for x86_64 155 kB/s | 84 kB 00:00 2024-09-18T07:11:40.739 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 49 kB/s | 24 kB 00:00 2024-09-18T07:11:40.784 INFO:teuthology.orchestra.run.smithi172.stdout:ceph noarch packages 48 kB/s | 24 kB 00:00 2024-09-18T07:11:41.290 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2024-09-18T07:11:41.326 INFO:teuthology.orchestra.run.smithi172.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-09-18T07:11:43.309 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.3 MB 00:01 2024-09-18T07:11:44.298 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.3 MB 00:02 2024-09-18T07:11:47.416 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-18T07:11:52.626 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 20 MB 00:08 2024-09-18T07:11:53.024 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-18T07:11:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 19 kB 00:00 2024-09-18T07:11:56.031 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-18T07:11:57.868 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-18T07:11:59.236 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 133 kB/s | 19 kB 00:00 2024-09-18T07:12:00.540 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-18T07:12:01.412 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-18T07:12:03.647 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-18T07:12:03.647 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-18T07:12:03.708 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-18T07:12:03.725 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-09-18T07:12:03.725 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-18T07:12:03.725 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-09-18T07:12:03.725 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-18T07:12:03.725 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 6.6 k 2024-09-18T07:12:03.725 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 5.5 M 2024-09-18T07:12:03.725 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 900 k 2024-09-18T07:12:03.726 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 145 k 2024-09-18T07:12:03.726 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 1.5 M 2024-09-18T07:12:03.726 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 154 k 2024-09-18T07:12:03.726 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 14 M 2024-09-18T07:12:03.726 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 7.4 M 2024-09-18T07:12:03.726 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 49 k 2024-09-18T07:12:03.726 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 11 M 2024-09-18T07:12:03.726 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 45 M 2024-09-18T07:12:03.726 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 285 k 2024-09-18T07:12:03.726 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 784 k 2024-09-18T07:12:03.726 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 32 k 2024-09-18T07:12:03.727 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 755 k 2024-09-18T07:12:03.727 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 126 k 2024-09-18T07:12:03.727 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 161 k 2024-09-18T07:12:03.727 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 320 k 2024-09-18T07:12:03.727 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 305 k 2024-09-18T07:12:03.727 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 99 k 2024-09-18T07:12:03.727 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 86 k 2024-09-18T07:12:03.727 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 3.2 M 2024-09-18T07:12:03.727 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 171 k 2024-09-18T07:12:03.727 INFO:teuthology.orchestra.run.smithi070.stdout: sqlite-devel x86_64 3.34.1-7.el9 appstream 134 k 2024-09-18T07:12:03.727 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2024-09-18T07:12:03.727 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 3.5 M 2024-09-18T07:12:03.727 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 3.2 M 2024-09-18T07:12:03.728 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-18T07:12:03.728 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-18T07:12:03.728 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-18T07:12:03.728 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 22 M 2024-09-18T07:12:03.728 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 34 k 2024-09-18T07:12:03.728 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 2.4 M 2024-09-18T07:12:03.728 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 291 k 2024-09-18T07:12:03.728 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 4.8 M 2024-09-18T07:12:03.728 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 17 M 2024-09-18T07:12:03.728 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 16 k 2024-09-18T07:12:03.729 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 25 k 2024-09-18T07:12:03.729 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-18T07:12:03.729 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-18T07:12:03.729 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-18T07:12:03.729 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-18T07:12:03.729 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-18T07:12:03.729 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-18T07:12:03.729 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-18T07:12:03.729 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-18T07:12:03.729 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 161 k 2024-09-18T07:12:03.729 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-18T07:12:03.729 INFO:teuthology.orchestra.run.smithi070.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-18T07:12:03.730 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-18T07:12:03.730 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-18T07:12:03.730 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-18T07:12:03.730 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-18T07:12:03.730 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 508 k 2024-09-18T07:12:03.730 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-18T07:12:03.730 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 5.5 M 2024-09-18T07:12:03.730 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-18T07:12:03.730 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-18T07:12:03.730 INFO:teuthology.orchestra.run.smithi070.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-18T07:12:03.730 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-18T07:12:03.730 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-18T07:12:03.731 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-18T07:12:03.731 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-18T07:12:03.731 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-18T07:12:03.731 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-18T07:12:03.731 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-18T07:12:03.731 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-18T07:12:03.731 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-18T07:12:03.731 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-18T07:12:03.731 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-18T07:12:03.731 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 45 k 2024-09-18T07:12:03.731 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 143 k 2024-09-18T07:12:03.731 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-18T07:12:03.732 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-18T07:12:03.732 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-18T07:12:03.732 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-18T07:12:03.732 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-18T07:12:03.732 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-18T07:12:03.732 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-18T07:12:03.732 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-18T07:12:03.732 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-18T07:12:03.732 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-18T07:12:03.732 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-18T07:12:03.732 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-18T07:12:03.732 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-18T07:12:03.732 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-18T07:12:03.733 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-18T07:12:03.733 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-18T07:12:03.733 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-18T07:12:03.733 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-18T07:12:03.733 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-18T07:12:03.733 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-18T07:12:03.733 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-18T07:12:03.733 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-18T07:12:03.733 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-18T07:12:03.733 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-18T07:12:03.733 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-18T07:12:03.733 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-18T07:12:03.734 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-18T07:12:03.734 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-18T07:12:03.734 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-18T07:12:03.734 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-18T07:12:03.734 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-18T07:12:03.734 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-18T07:12:03.734 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-18T07:12:03.734 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-18T07:12:03.734 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-18T07:12:03.734 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-18T07:12:03.734 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-18T07:12:03.734 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-18T07:12:03.734 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-18T07:12:03.735 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-18T07:12:03.735 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-18T07:12:03.735 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-18T07:12:03.735 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-18T07:12:03.735 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-18T07:12:03.735 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-18T07:12:03.735 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-18T07:12:03.735 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-18T07:12:03.735 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-18T07:12:03.735 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-18T07:12:03.735 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-18T07:12:03.735 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-18T07:12:03.736 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-18T07:12:03.736 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-18T07:12:03.736 INFO:teuthology.orchestra.run.smithi070.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-18T07:12:03.736 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-18T07:12:03.736 INFO:teuthology.orchestra.run.smithi070.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-09-18T07:12:03.736 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-18T07:12:03.736 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-18T07:12:03.736 INFO:teuthology.orchestra.run.smithi070.stdout:Installing weak dependencies: 2024-09-18T07:12:03.736 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-18T07:12:03.736 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:12:03.736 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-18T07:12:03.736 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-09-18T07:12:03.737 INFO:teuthology.orchestra.run.smithi070.stdout:Install 127 Packages 2024-09-18T07:12:03.737 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 2 Packages 2024-09-18T07:12:03.737 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:12:03.737 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 212 M 2024-09-18T07:12:03.738 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-18T07:12:04.454 INFO:teuthology.orchestra.run.smithi070.stdout:(1/129): ceph-19.3.0-4953.g82e3d596.el9.x86_64. 53 kB/s | 6.6 kB 00:00 2024-09-18T07:12:04.854 INFO:teuthology.orchestra.run.smithi070.stdout:(2/129): ceph-fuse-19.3.0-4953.g82e3d596.el9.x8 2.2 MB/s | 900 kB 00:00 2024-09-18T07:12:05.622 INFO:teuthology.orchestra.run.smithi070.stdout:(3/129): ceph-common-19.3.0-4953.g82e3d596.el9. 17 MB/s | 22 MB 00:01 2024-09-18T07:12:05.655 INFO:teuthology.orchestra.run.smithi070.stdout:(4/129): ceph-immutable-object-cache-19.3.0-495 181 kB/s | 145 kB 00:00 2024-09-18T07:12:05.739 INFO:teuthology.orchestra.run.smithi070.stdout:(5/129): ceph-mds-19.3.0-4953.g82e3d596.el9.x86 20 MB/s | 2.4 MB 00:00 2024-09-18T07:12:05.806 INFO:teuthology.orchestra.run.smithi070.stdout:(6/129): ceph-base-19.3.0-4953.g82e3d596.el9.x8 3.7 MB/s | 5.5 MB 00:01 2024-09-18T07:12:05.864 INFO:teuthology.orchestra.run.smithi070.stdout:(7/129): ceph-mgr-19.3.0-4953.g82e3d596.el9.x86 7.3 MB/s | 1.5 MB 00:00 2024-09-18T07:12:05.948 INFO:teuthology.orchestra.run.smithi070.stdout:(8/129): ceph-mon-19.3.0-4953.g82e3d596.el9.x86 23 MB/s | 4.8 MB 00:00 2024-09-18T07:12:05.962 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-18T07:12:05.998 INFO:teuthology.orchestra.run.smithi070.stdout:(9/129): ceph-selinux-19.3.0-4953.g82e3d596.el9 506 kB/s | 25 kB 00:00 2024-09-18T07:12:08.048 INFO:teuthology.orchestra.run.smithi070.stdout:(10/129): ceph-test-19.3.0-4953.g82e3d596.el9.x 22 MB/s | 45 MB 00:02 2024-09-18T07:12:08.115 INFO:teuthology.orchestra.run.smithi070.stdout:(11/129): libcephfs-devel-19.3.0-4953.g82e3d596 480 kB/s | 32 kB 00:00 2024-09-18T07:12:08.156 INFO:teuthology.orchestra.run.smithi172.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-18T07:12:08.156 INFO:teuthology.orchestra.run.smithi172.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-18T07:12:08.216 INFO:teuthology.orchestra.run.smithi070.stdout:(12/129): libcephfs2-19.3.0-4953.g82e3d596.el9. 7.4 MB/s | 755 kB 00:00 2024-09-18T07:12:08.217 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-18T07:12:08.246 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-09-18T07:12:08.246 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-18T07:12:08.246 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-09-18T07:12:08.246 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-18T07:12:08.246 INFO:teuthology.orchestra.run.smithi172.stdout: ceph x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 6.6 k 2024-09-18T07:12:08.246 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 5.5 M 2024-09-18T07:12:08.246 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 900 k 2024-09-18T07:12:08.246 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 145 k 2024-09-18T07:12:08.247 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 1.5 M 2024-09-18T07:12:08.247 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 154 k 2024-09-18T07:12:08.247 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 14 M 2024-09-18T07:12:08.247 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 7.4 M 2024-09-18T07:12:08.247 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 49 k 2024-09-18T07:12:08.247 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 11 M 2024-09-18T07:12:08.247 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 45 M 2024-09-18T07:12:08.247 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 285 k 2024-09-18T07:12:08.247 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 784 k 2024-09-18T07:12:08.247 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 32 k 2024-09-18T07:12:08.247 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2 x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 755 k 2024-09-18T07:12:08.247 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 126 k 2024-09-18T07:12:08.248 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 161 k 2024-09-18T07:12:08.248 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 320 k 2024-09-18T07:12:08.248 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 305 k 2024-09-18T07:12:08.248 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 99 k 2024-09-18T07:12:08.248 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 86 k 2024-09-18T07:12:08.248 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 3.2 M 2024-09-18T07:12:08.248 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 171 k 2024-09-18T07:12:08.248 INFO:teuthology.orchestra.run.smithi172.stdout: sqlite-devel x86_64 3.34.1-7.el9 appstream 134 k 2024-09-18T07:12:08.248 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrading: 2024-09-18T07:12:08.248 INFO:teuthology.orchestra.run.smithi172.stdout: librados2 x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 3.5 M 2024-09-18T07:12:08.248 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1 x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 3.2 M 2024-09-18T07:12:08.248 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-18T07:12:08.248 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-18T07:12:08.249 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-18T07:12:08.249 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 22 M 2024-09-18T07:12:08.249 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 34 k 2024-09-18T07:12:08.249 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 2.4 M 2024-09-18T07:12:08.249 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 291 k 2024-09-18T07:12:08.249 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 4.8 M 2024-09-18T07:12:08.249 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 17 M 2024-09-18T07:12:08.249 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4953.g82e3d596.el9 ceph-noarch 16 k 2024-09-18T07:12:08.249 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 25 k 2024-09-18T07:12:08.249 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-18T07:12:08.249 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-18T07:12:08.249 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-18T07:12:08.249 INFO:teuthology.orchestra.run.smithi172.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-18T07:12:08.250 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-18T07:12:08.250 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-18T07:12:08.250 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-18T07:12:08.250 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-18T07:12:08.250 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 161 k 2024-09-18T07:12:08.250 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-18T07:12:08.250 INFO:teuthology.orchestra.run.smithi172.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-18T07:12:08.250 INFO:teuthology.orchestra.run.smithi172.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-18T07:12:08.250 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-18T07:12:08.250 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-18T07:12:08.250 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-18T07:12:08.250 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 508 k 2024-09-18T07:12:08.250 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-18T07:12:08.251 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2 x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 5.5 M 2024-09-18T07:12:08.251 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-18T07:12:08.251 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-18T07:12:08.251 INFO:teuthology.orchestra.run.smithi172.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-18T07:12:08.251 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-18T07:12:08.251 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-18T07:12:08.251 INFO:teuthology.orchestra.run.smithi172.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-18T07:12:08.251 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-18T07:12:08.251 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-18T07:12:08.251 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-18T07:12:08.251 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-18T07:12:08.251 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-18T07:12:08.251 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 45 k 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common x86_64 2:19.3.0-4953.g82e3d596.el9 ceph 143 k 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-18T07:12:08.252 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-18T07:12:08.253 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-18T07:12:08.253 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-18T07:12:08.253 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-18T07:12:08.253 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-18T07:12:08.253 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-18T07:12:08.253 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-18T07:12:08.253 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-18T07:12:08.253 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-18T07:12:08.253 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-18T07:12:08.253 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-18T07:12:08.253 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-18T07:12:08.253 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-18T07:12:08.254 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-18T07:12:08.254 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-18T07:12:08.254 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-18T07:12:08.254 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-18T07:12:08.254 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-18T07:12:08.254 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-18T07:12:08.254 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-18T07:12:08.254 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-18T07:12:08.254 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-18T07:12:08.254 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-18T07:12:08.254 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-18T07:12:08.254 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-18T07:12:08.254 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-18T07:12:08.255 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-18T07:12:08.255 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-18T07:12:08.255 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-18T07:12:08.255 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-18T07:12:08.255 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-18T07:12:08.255 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-18T07:12:08.255 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-18T07:12:08.255 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-18T07:12:08.255 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-18T07:12:08.255 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-18T07:12:08.255 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-18T07:12:08.255 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-18T07:12:08.255 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout:Installing weak dependencies: 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:12:08.256 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-18T07:12:08.257 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-09-18T07:12:08.257 INFO:teuthology.orchestra.run.smithi172.stdout:Install 127 Packages 2024-09-18T07:12:08.257 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrade 2 Packages 2024-09-18T07:12:08.257 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:12:08.257 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 212 M 2024-09-18T07:12:08.257 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-18T07:12:08.633 INFO:teuthology.orchestra.run.smithi070.stdout:(13/129): ceph-osd-19.3.0-4953.g82e3d596.el9.x8 6.1 MB/s | 17 MB 00:02 2024-09-18T07:12:08.658 INFO:teuthology.orchestra.run.smithi070.stdout:(14/129): libcephsqlite-19.3.0-4953.g82e3d596.e 366 kB/s | 161 kB 00:00 2024-09-18T07:12:08.800 INFO:teuthology.orchestra.run.smithi070.stdout:(15/129): ceph-radosgw-19.3.0-4953.g82e3d596.el 3.7 MB/s | 11 MB 00:02 2024-09-18T07:12:08.825 INFO:teuthology.orchestra.run.smithi070.stdout:(16/129): librados-devel-19.3.0-4953.g82e3d596. 659 kB/s | 126 kB 00:00 2024-09-18T07:12:09.097 INFO:teuthology.orchestra.run.smithi172.stdout:(1/129): ceph-19.3.0-4953.g82e3d596.el9.x86_64. 53 kB/s | 6.6 kB 00:00 2024-09-18T07:12:09.347 INFO:teuthology.orchestra.run.smithi172.stdout:(2/129): ceph-fuse-19.3.0-4953.g82e3d596.el9.x8 3.5 MB/s | 900 kB 00:00 2024-09-18T07:12:09.408 INFO:teuthology.orchestra.run.smithi070.stdout:(17/129): libradosstriper1-19.3.0-4953.g82e3d59 677 kB/s | 508 kB 00:00 2024-09-18T07:12:09.448 INFO:teuthology.orchestra.run.smithi172.stdout:(3/129): ceph-base-19.3.0-4953.g82e3d596.el9.x8 12 MB/s | 5.5 MB 00:00 2024-09-18T07:12:09.467 INFO:teuthology.orchestra.run.smithi070.stdout:(18/129): python3-ceph-argparse-19.3.0-4953.g82 70 kB/s | 45 kB 00:00 2024-09-18T07:12:09.473 INFO:teuthology.orchestra.run.smithi172.stdout:(4/129): ceph-immutable-object-cache-19.3.0-495 1.1 MB/s | 145 kB 00:00 2024-09-18T07:12:09.493 INFO:teuthology.orchestra.run.smithi070.stdout:(19/129): python3-ceph-common-19.3.0-4953.g82e3 1.7 MB/s | 143 kB 00:00 2024-09-18T07:12:09.598 INFO:teuthology.orchestra.run.smithi172.stdout:(5/129): ceph-mds-19.3.0-4953.g82e3d596.el9.x86 16 MB/s | 2.4 MB 00:00 2024-09-18T07:12:09.601 INFO:teuthology.orchestra.run.smithi070.stdout:(20/129): python3-cephfs-19.3.0-4953.g82e3d596. 1.2 MB/s | 161 kB 00:00 2024-09-18T07:12:09.626 INFO:teuthology.orchestra.run.smithi070.stdout:(21/129): python3-rados-19.3.0-4953.g82e3d596.e 2.3 MB/s | 320 kB 00:00 2024-09-18T07:12:09.768 INFO:teuthology.orchestra.run.smithi070.stdout:(22/129): python3-rgw-19.3.0-4953.g82e3d596.el9 702 kB/s | 99 kB 00:00 2024-09-18T07:12:09.835 INFO:teuthology.orchestra.run.smithi070.stdout:(23/129): rbd-fuse-19.3.0-4953.g82e3d596.el9.x8 1.3 MB/s | 86 kB 00:00 2024-09-18T07:12:09.861 INFO:teuthology.orchestra.run.smithi070.stdout:(24/129): python3-rbd-19.3.0-4953.g82e3d596.el9 1.2 MB/s | 305 kB 00:00 2024-09-18T07:12:09.881 INFO:teuthology.orchestra.run.smithi172.stdout:(6/129): ceph-mon-19.3.0-4953.g82e3d596.el9.x86 17 MB/s | 4.8 MB 00:00 2024-09-18T07:12:09.915 INFO:teuthology.orchestra.run.smithi172.stdout:(7/129): ceph-mgr-19.3.0-4953.g82e3d596.el9.x86 3.5 MB/s | 1.5 MB 00:00 2024-09-18T07:12:09.977 INFO:teuthology.orchestra.run.smithi070.stdout:(25/129): librgw2-19.3.0-4953.g82e3d596.el9.x86 4.7 MB/s | 5.5 MB 00:01 2024-09-18T07:12:10.095 INFO:teuthology.orchestra.run.smithi070.stdout:(26/129): rbd-nbd-19.3.0-4953.g82e3d596.el9.x86 731 kB/s | 171 kB 00:00 2024-09-18T07:12:10.128 INFO:teuthology.orchestra.run.smithi070.stdout:(27/129): ceph-grafana-dashboards-19.3.0-4953.g 228 kB/s | 34 kB 00:00 2024-09-18T07:12:10.170 INFO:teuthology.orchestra.run.smithi070.stdout:(28/129): ceph-mgr-cephadm-19.3.0-4953.g82e3d59 2.0 MB/s | 154 kB 00:00 2024-09-18T07:12:10.229 INFO:teuthology.orchestra.run.smithi070.stdout:(29/129): rbd-mirror-19.3.0-4953.g82e3d596.el9. 8.1 MB/s | 3.2 MB 00:00 2024-09-18T07:12:10.257 INFO:teuthology.orchestra.run.smithi172.stdout:(8/129): ceph-common-19.3.0-4953.g82e3d596.el9. 17 MB/s | 22 MB 00:01 2024-09-18T07:12:10.299 INFO:teuthology.orchestra.run.smithi172.stdout:(9/129): ceph-selinux-19.3.0-4953.g82e3d596.el9 608 kB/s | 25 kB 00:00 2024-09-18T07:12:10.321 INFO:teuthology.orchestra.run.smithi070.stdout:(30/129): ceph-mgr-modules-core-19.3.0-4953.g82 3.1 MB/s | 291 kB 00:00 2024-09-18T07:12:10.404 INFO:teuthology.orchestra.run.smithi070.stdout:(31/129): ceph-mgr-rook-19.3.0-4953.g82e3d596.e 592 kB/s | 49 kB 00:00 2024-09-18T07:12:10.504 INFO:teuthology.orchestra.run.smithi070.stdout:(32/129): ceph-prometheus-alerts-19.3.0-4953.g8 164 kB/s | 16 kB 00:00 2024-09-18T07:12:10.646 INFO:teuthology.orchestra.run.smithi070.stdout:(33/129): ceph-volume-19.3.0-4953.g82e3d596.el9 2.0 MB/s | 285 kB 00:00 2024-09-18T07:12:10.822 INFO:teuthology.orchestra.run.smithi070.stdout:(34/129): cephadm-19.3.0-4953.g82e3d596.el9.noa 4.4 MB/s | 784 kB 00:00 2024-09-18T07:12:10.832 INFO:teuthology.orchestra.run.smithi172.stdout:(10/129): ceph-osd-19.3.0-4953.g82e3d596.el9.x8 18 MB/s | 17 MB 00:00 2024-09-18T07:12:10.883 INFO:teuthology.orchestra.run.smithi172.stdout:(11/129): libcephfs-devel-19.3.0-4953.g82e3d596 638 kB/s | 32 kB 00:00 2024-09-18T07:12:10.930 INFO:teuthology.orchestra.run.smithi070.stdout:(35/129): fuse-2.9.9-16.el9.x86_64.rpm 737 kB/s | 80 kB 00:00 2024-09-18T07:12:10.983 INFO:teuthology.orchestra.run.smithi172.stdout:(12/129): libcephfs2-19.3.0-4953.g82e3d596.el9. 7.4 MB/s | 755 kB 00:00 2024-09-18T07:12:11.031 INFO:teuthology.orchestra.run.smithi070.stdout:(36/129): libgfortran-11.5.0-2.el9.x86_64.rpm 7.8 MB/s | 801 kB 00:00 2024-09-18T07:12:11.075 INFO:teuthology.orchestra.run.smithi172.stdout:(13/129): libcephsqlite-19.3.0-4953.g82e3d596.e 1.7 MB/s | 161 kB 00:00 2024-09-18T07:12:11.114 INFO:teuthology.orchestra.run.smithi070.stdout:(37/129): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-18T07:12:11.158 INFO:teuthology.orchestra.run.smithi172.stdout:(14/129): librados-devel-19.3.0-4953.g82e3d596. 1.5 MB/s | 126 kB 00:00 2024-09-18T07:12:11.214 INFO:teuthology.orchestra.run.smithi070.stdout:(38/129): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-18T07:12:11.250 INFO:teuthology.orchestra.run.smithi172.stdout:(15/129): libradosstriper1-19.3.0-4953.g82e3d59 5.4 MB/s | 508 kB 00:00 2024-09-18T07:12:11.557 INFO:teuthology.orchestra.run.smithi070.stdout:(39/129): python3-cryptography-36.0.1-4.el9.x86 3.6 MB/s | 1.2 MB 00:00 2024-09-18T07:12:11.615 INFO:teuthology.orchestra.run.smithi070.stdout:(40/129): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-18T07:12:11.617 INFO:teuthology.orchestra.run.smithi172.stdout:(16/129): librgw2-19.3.0-4953.g82e3d596.el9.x86 15 MB/s | 5.5 MB 00:00 2024-09-18T07:12:11.757 INFO:teuthology.orchestra.run.smithi070.stdout:(41/129): ceph-mgr-dashboard-19.3.0-4953.g82e3d 8.7 MB/s | 14 MB 00:01 2024-09-18T07:12:11.768 INFO:teuthology.orchestra.run.smithi172.stdout:(17/129): ceph-radosgw-19.3.0-4953.g82e3d596.el 5.9 MB/s | 11 MB 00:01 2024-09-18T07:12:11.782 INFO:teuthology.orchestra.run.smithi070.stdout:(42/129): python3-pycparser-2.20-6.el9.noarch.r 810 kB/s | 135 kB 00:00 2024-09-18T07:12:11.784 INFO:teuthology.orchestra.run.smithi172.stdout:(18/129): python3-ceph-argparse-19.3.0-4953.g82 270 kB/s | 45 kB 00:00 2024-09-18T07:12:11.818 INFO:teuthology.orchestra.run.smithi172.stdout:(19/129): python3-ceph-common-19.3.0-4953.g82e3 2.8 MB/s | 143 kB 00:00 2024-09-18T07:12:11.843 INFO:teuthology.orchestra.run.smithi172.stdout:(20/129): python3-cephfs-19.3.0-4953.g82e3d596. 2.7 MB/s | 161 kB 00:00 2024-09-18T07:12:11.869 INFO:teuthology.orchestra.run.smithi172.stdout:(21/129): python3-rados-19.3.0-4953.g82e3d596.e 6.3 MB/s | 320 kB 00:00 2024-09-18T07:12:11.883 INFO:teuthology.orchestra.run.smithi070.stdout:(43/129): ceph-mgr-diskprediction-local-19.3.0- 4.3 MB/s | 7.4 MB 00:01 2024-09-18T07:12:11.894 INFO:teuthology.orchestra.run.smithi172.stdout:(22/129): python3-rbd-19.3.0-4953.g82e3d596.el9 6.0 MB/s | 305 kB 00:00 2024-09-18T07:12:11.908 INFO:teuthology.orchestra.run.smithi070.stdout:(44/129): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-09-18T07:12:11.919 INFO:teuthology.orchestra.run.smithi172.stdout:(23/129): python3-rgw-19.3.0-4953.g82e3d596.el9 1.9 MB/s | 99 kB 00:00 2024-09-18T07:12:11.933 INFO:teuthology.orchestra.run.smithi070.stdout:(45/129): python3-requests-2.25.1-8.el9.noarch. 711 kB/s | 125 kB 00:00 2024-09-18T07:12:11.944 INFO:teuthology.orchestra.run.smithi172.stdout:(24/129): rbd-fuse-19.3.0-4953.g82e3d596.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-18T07:12:11.959 INFO:teuthology.orchestra.run.smithi070.stdout:(46/129): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-18T07:12:12.001 INFO:teuthology.orchestra.run.smithi070.stdout:(47/129): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-18T07:12:12.011 INFO:teuthology.orchestra.run.smithi172.stdout:(25/129): rbd-nbd-19.3.0-4953.g82e3d596.el9.x86 2.5 MB/s | 171 kB 00:00 2024-09-18T07:12:12.026 INFO:teuthology.orchestra.run.smithi070.stdout:(48/129): boost-program-options-1.75.0-8.el9.x8 749 kB/s | 107 kB 00:00 2024-09-18T07:12:12.051 INFO:teuthology.orchestra.run.smithi070.stdout:(49/129): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-18T07:12:12.070 INFO:teuthology.orchestra.run.smithi172.stdout:(26/129): ceph-grafana-dashboards-19.3.0-4953.g 588 kB/s | 34 kB 00:00 2024-09-18T07:12:12.085 INFO:teuthology.orchestra.run.smithi070.stdout:(50/129): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-18T07:12:12.110 INFO:teuthology.orchestra.run.smithi070.stdout:(51/129): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-09-18T07:12:12.152 INFO:teuthology.orchestra.run.smithi070.stdout:(52/129): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-09-18T07:12:12.202 INFO:teuthology.orchestra.run.smithi070.stdout:(53/129): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-18T07:12:12.236 INFO:teuthology.orchestra.run.smithi070.stdout:(54/129): openblas-0.3.26-2.el9.x86_64.rpm 1.1 MB/s | 38 kB 00:00 2024-09-18T07:12:12.261 INFO:teuthology.orchestra.run.smithi070.stdout:(55/129): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-18T07:12:12.521 INFO:teuthology.orchestra.run.smithi172.stdout:(27/129): ceph-test-19.3.0-4953.g82e3d596.el9.x 20 MB/s | 45 MB 00:02 2024-09-18T07:12:12.546 INFO:teuthology.orchestra.run.smithi172.stdout:(28/129): ceph-mgr-cephadm-19.3.0-4953.g82e3d59 325 kB/s | 154 kB 00:00 2024-09-18T07:12:12.553 INFO:teuthology.orchestra.run.smithi070.stdout:(56/129): protobuf-3.14.0-13.el9.x86_64.rpm 3.5 MB/s | 1.0 MB 00:00 2024-09-18T07:12:12.604 INFO:teuthology.orchestra.run.smithi172.stdout:(29/129): rbd-mirror-19.3.0-4953.g82e3d596.el9. 4.7 MB/s | 3.2 MB 00:00 2024-09-18T07:12:12.637 INFO:teuthology.orchestra.run.smithi070.stdout:(57/129): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.3 MB/s | 3.0 MB 00:00 2024-09-18T07:12:12.729 INFO:teuthology.orchestra.run.smithi070.stdout:(58/129): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-18T07:12:12.755 INFO:teuthology.orchestra.run.smithi172.stdout:(30/129): ceph-mgr-modules-core-19.3.0-4953.g82 1.9 MB/s | 291 kB 00:00 2024-09-18T07:12:12.821 INFO:teuthology.orchestra.run.smithi070.stdout:(59/129): python3-jinja2-2.11.3-6.el9.noarch.rp 2.7 MB/s | 248 kB 00:00 2024-09-18T07:12:12.855 INFO:teuthology.orchestra.run.smithi172.stdout:(31/129): ceph-mgr-rook-19.3.0-4953.g82e3d596.e 494 kB/s | 49 kB 00:00 2024-09-18T07:12:12.862 INFO:teuthology.orchestra.run.smithi070.stdout:(60/129): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-09-18T07:12:12.905 INFO:teuthology.orchestra.run.smithi070.stdout:(61/129): python3-jsonpointer-2.0-4.el9.noarch. 460 kB/s | 19 kB 00:00 2024-09-18T07:12:12.963 INFO:teuthology.orchestra.run.smithi172.stdout:(32/129): ceph-mgr-diskprediction-local-19.3.0- 18 MB/s | 7.4 MB 00:00 2024-09-18T07:12:12.971 INFO:teuthology.orchestra.run.smithi070.stdout:(62/129): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-18T07:12:12.980 INFO:teuthology.orchestra.run.smithi172.stdout:(33/129): ceph-prometheus-alerts-19.3.0-4953.g8 131 kB/s | 16 kB 00:00 2024-09-18T07:12:13.014 INFO:teuthology.orchestra.run.smithi070.stdout:(63/129): python3-markupsafe-1.1.1-12.el9.x86_6 834 kB/s | 35 kB 00:00 2024-09-18T07:12:13.014 INFO:teuthology.orchestra.run.smithi172.stdout:(34/129): ceph-volume-19.3.0-4953.g82e3d596.el9 4.8 MB/s | 285 kB 00:00 2024-09-18T07:12:13.173 INFO:teuthology.orchestra.run.smithi172.stdout:(35/129): ceph-mgr-dashboard-19.3.0-4953.g82e3d 22 MB/s | 14 MB 00:00 2024-09-18T07:12:13.215 INFO:teuthology.orchestra.run.smithi172.stdout:(36/129): cephadm-19.3.0-4953.g82e3d596.el9.noa 3.3 MB/s | 784 kB 00:00 2024-09-18T07:12:13.257 INFO:teuthology.orchestra.run.smithi172.stdout:(37/129): fuse-2.9.9-16.el9.x86_64.rpm 330 kB/s | 80 kB 00:00 2024-09-18T07:12:13.323 INFO:teuthology.orchestra.run.smithi172.stdout:(38/129): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-18T07:12:13.349 INFO:teuthology.orchestra.run.smithi172.stdout:(39/129): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-18T07:12:13.391 INFO:teuthology.orchestra.run.smithi172.stdout:(40/129): libgfortran-11.5.0-2.el9.x86_64.rpm 3.6 MB/s | 801 kB 00:00 2024-09-18T07:12:13.416 INFO:teuthology.orchestra.run.smithi172.stdout:(41/129): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-18T07:12:13.450 INFO:teuthology.orchestra.run.smithi172.stdout:(42/129): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-18T07:12:13.475 INFO:teuthology.orchestra.run.smithi172.stdout:(43/129): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-18T07:12:13.500 INFO:teuthology.orchestra.run.smithi172.stdout:(44/129): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-18T07:12:13.542 INFO:teuthology.orchestra.run.smithi172.stdout:(45/129): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-18T07:12:13.609 INFO:teuthology.orchestra.run.smithi172.stdout:(46/129): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-18T07:12:13.634 INFO:teuthology.orchestra.run.smithi172.stdout:(47/129): boost-program-options-1.75.0-8.el9.x8 671 kB/s | 107 kB 00:00 2024-09-18T07:12:13.651 INFO:teuthology.orchestra.run.smithi172.stdout:(48/129): flexiblas-openblas-openmp-3.0.4-8.el9 435 kB/s | 18 kB 00:00 2024-09-18T07:12:13.672 INFO:teuthology.orchestra.run.smithi070.stdout:(64/129): openblas-openmp-0.3.26-2.el9.x86_64.r 3.4 MB/s | 4.9 MB 00:01 2024-09-18T07:12:13.718 INFO:teuthology.orchestra.run.smithi172.stdout:(49/129): libnbd-1.20.2-2.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2024-09-18T07:12:13.744 INFO:teuthology.orchestra.run.smithi172.stdout:(50/129): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-18T07:12:13.761 INFO:teuthology.orchestra.run.smithi172.stdout:(51/129): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-18T07:12:13.764 INFO:teuthology.orchestra.run.smithi070.stdout:(65/129): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-18T07:12:13.819 INFO:teuthology.orchestra.run.smithi172.stdout:(52/129): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-18T07:12:13.853 INFO:teuthology.orchestra.run.smithi172.stdout:(53/129): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-18T07:12:13.856 INFO:teuthology.orchestra.run.smithi070.stdout:(66/129): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-18T07:12:13.878 INFO:teuthology.orchestra.run.smithi172.stdout:(54/129): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-09-18T07:12:13.903 INFO:teuthology.orchestra.run.smithi172.stdout:(55/129): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-18T07:12:13.915 INFO:teuthology.orchestra.run.smithi070.stdout:(67/129): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-18T07:12:13.928 INFO:teuthology.orchestra.run.smithi172.stdout:(56/129): openblas-0.3.26-2.el9.x86_64.rpm 512 kB/s | 38 kB 00:00 2024-09-18T07:12:13.957 INFO:teuthology.orchestra.run.smithi070.stdout:(68/129): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-18T07:12:14.004 INFO:teuthology.orchestra.run.smithi172.stdout:(57/129): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-18T07:12:14.057 INFO:teuthology.orchestra.run.smithi070.stdout:(69/129): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-09-18T07:12:14.079 INFO:teuthology.orchestra.run.smithi172.stdout:(58/129): openblas-openmp-0.3.26-2.el9.x86_64.r 25 MB/s | 4.9 MB 00:00 2024-09-18T07:12:14.104 INFO:teuthology.orchestra.run.smithi172.stdout:(59/129): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-18T07:12:14.129 INFO:teuthology.orchestra.run.smithi172.stdout:(60/129): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-18T07:12:14.132 INFO:teuthology.orchestra.run.smithi070.stdout:(70/129): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-18T07:12:14.155 INFO:teuthology.orchestra.run.smithi172.stdout:(61/129): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-18T07:12:14.180 INFO:teuthology.orchestra.run.smithi172.stdout:(62/129): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-18T07:12:14.205 INFO:teuthology.orchestra.run.smithi172.stdout:(63/129): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-18T07:12:14.231 INFO:teuthology.orchestra.run.smithi172.stdout:(64/129): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-18T07:12:14.241 INFO:teuthology.orchestra.run.smithi070.stdout:(71/129): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-09-18T07:12:14.283 INFO:teuthology.orchestra.run.smithi070.stdout:(72/129): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-18T07:12:14.306 INFO:teuthology.orchestra.run.smithi172.stdout:(65/129): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-18T07:12:14.325 INFO:teuthology.orchestra.run.smithi070.stdout:(73/129): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-18T07:12:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:(66/129): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-18T07:12:14.357 INFO:teuthology.orchestra.run.smithi172.stdout:(67/129): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-18T07:12:14.382 INFO:teuthology.orchestra.run.smithi172.stdout:(68/129): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-18T07:12:14.407 INFO:teuthology.orchestra.run.smithi172.stdout:(69/129): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-09-18T07:12:14.441 INFO:teuthology.orchestra.run.smithi172.stdout:(70/129): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-18T07:12:14.466 INFO:teuthology.orchestra.run.smithi172.stdout:(71/129): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-18T07:12:14.491 INFO:teuthology.orchestra.run.smithi172.stdout:(72/129): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-18T07:12:14.492 INFO:teuthology.orchestra.run.smithi070.stdout:(74/129): python3-babel-2.9.1-2.el9.noarch.rpm 3.1 MB/s | 6.0 MB 00:01 2024-09-18T07:12:14.516 INFO:teuthology.orchestra.run.smithi172.stdout:(73/129): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-18T07:12:14.542 INFO:teuthology.orchestra.run.smithi172.stdout:(74/129): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-18T07:12:14.559 INFO:teuthology.orchestra.run.smithi070.stdout:(75/129): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.3 MB/s | 5.1 MB 00:01 2024-09-18T07:12:14.584 INFO:teuthology.orchestra.run.smithi172.stdout:(75/129): python3-toml-0.10.2-6.el9.noarch.rpm 998 kB/s | 42 kB 00:00 2024-09-18T07:12:14.584 INFO:teuthology.orchestra.run.smithi070.stdout:(76/129): python3-toml-0.10.2-6.el9.noarch.rpm 454 kB/s | 42 kB 00:00 2024-09-18T07:12:14.626 INFO:teuthology.orchestra.run.smithi070.stdout:(77/129): qatlib-service-24.02.0-1.el9.x86_64.r 864 kB/s | 36 kB 00:00 2024-09-18T07:12:14.650 INFO:teuthology.orchestra.run.smithi172.stdout:(76/129): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-18T07:12:14.676 INFO:teuthology.orchestra.run.smithi172.stdout:(77/129): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-09-18T07:12:14.677 INFO:teuthology.orchestra.run.smithi070.stdout:(78/129): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-18T07:12:14.701 INFO:teuthology.orchestra.run.smithi172.stdout:(78/129): qatlib-service-24.02.0-1.el9.x86_64.r 712 kB/s | 36 kB 00:00 2024-09-18T07:12:14.702 INFO:teuthology.orchestra.run.smithi070.stdout:(79/129): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-09-18T07:12:14.727 INFO:teuthology.orchestra.run.smithi172.stdout:(79/129): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-18T07:12:14.760 INFO:teuthology.orchestra.run.smithi172.stdout:(80/129): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-18T07:12:14.785 INFO:teuthology.orchestra.run.smithi070.stdout:(80/129): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-09-18T07:12:14.845 INFO:teuthology.orchestra.run.smithi172.stdout:(81/129): sqlite-3.34.1-7.el9.x86_64.rpm 6.2 MB/s | 748 kB 00:00 2024-09-18T07:12:14.860 INFO:teuthology.orchestra.run.smithi070.stdout:(81/129): sqlite-devel-3.34.1-7.el9.x86_64.rpm 1.8 MB/s | 134 kB 00:00 2024-09-18T07:12:14.869 INFO:teuthology.orchestra.run.smithi172.stdout:(82/129): sqlite-devel-3.34.1-7.el9.x86_64.rpm 1.2 MB/s | 134 kB 00:00 2024-09-18T07:12:14.894 INFO:teuthology.orchestra.run.smithi172.stdout:(83/129): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-09-18T07:12:14.919 INFO:teuthology.orchestra.run.smithi070.stdout:(82/129): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-18T07:12:14.986 INFO:teuthology.orchestra.run.smithi070.stdout:(83/129): sqlite-3.34.1-7.el9.x86_64.rpm 2.6 MB/s | 748 kB 00:00 2024-09-18T07:12:15.111 INFO:teuthology.orchestra.run.smithi172.stdout:(84/129): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-18T07:12:15.170 INFO:teuthology.orchestra.run.smithi172.stdout:(85/129): abseil-cpp-20211102.0-3.el9.x86_64.rp 9.2 MB/s | 548 kB 00:00 2024-09-18T07:12:15.203 INFO:teuthology.orchestra.run.smithi172.stdout:(86/129): gperftools-libs-2.9.1-3.el9.x86_64.rp 9.0 MB/s | 308 kB 00:00 2024-09-18T07:12:15.229 INFO:teuthology.orchestra.run.smithi172.stdout:(87/129): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-18T07:12:15.304 INFO:teuthology.orchestra.run.smithi172.stdout:(88/129): lua-devel-5.4.4-4.el9.x86_64.rpm 51 kB/s | 22 kB 00:00 2024-09-18T07:12:15.354 INFO:teuthology.orchestra.run.smithi172.stdout:(89/129): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-09-18T07:12:15.388 INFO:teuthology.orchestra.run.smithi172.stdout:(90/129): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-18T07:12:15.422 INFO:teuthology.orchestra.run.smithi172.stdout:(91/129): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-18T07:12:15.445 INFO:teuthology.orchestra.run.smithi070.stdout:(84/129): lua-devel-5.4.4-4.el9.x86_64.rpm 42 kB/s | 22 kB 00:00 2024-09-18T07:12:15.464 INFO:teuthology.orchestra.run.smithi172.stdout:(92/129): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-09-18T07:12:15.562 INFO:teuthology.orchestra.run.smithi070.stdout:(85/129): abseil-cpp-20211102.0-3.el9.x86_64.rp 4.6 MB/s | 548 kB 00:00 2024-09-18T07:12:15.597 INFO:teuthology.orchestra.run.smithi172.stdout:(93/129): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.2 MB/s | 839 kB 00:00 2024-09-18T07:12:15.628 INFO:teuthology.orchestra.run.smithi070.stdout:(86/129): gperftools-libs-2.9.1-3.el9.x86_64.rp 4.5 MB/s | 308 kB 00:00 2024-09-18T07:12:15.654 INFO:teuthology.orchestra.run.smithi070.stdout:(87/129): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-18T07:12:15.714 INFO:teuthology.orchestra.run.smithi172.stdout:(94/129): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-09-18T07:12:15.749 INFO:teuthology.orchestra.run.smithi172.stdout:(95/129): python3-asyncssh-2.13.2-5.el9.noarch. 3.6 MB/s | 548 kB 00:00 2024-09-18T07:12:15.773 INFO:teuthology.orchestra.run.smithi172.stdout:(96/129): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 744 kB/s | 43 kB 00:00 2024-09-18T07:12:15.798 INFO:teuthology.orchestra.run.smithi172.stdout:(97/129): python3-cachetools-4.2.4-1.el9.noarch 643 kB/s | 32 kB 00:00 2024-09-18T07:12:15.823 INFO:teuthology.orchestra.run.smithi172.stdout:(98/129): python3-certifi-2023.05.07-4.el9.noar 281 kB/s | 14 kB 00:00 2024-09-18T07:12:15.849 INFO:teuthology.orchestra.run.smithi172.stdout:(99/129): python3-cheroot-8.6.0-4.el9.noarch.rp 3.4 MB/s | 172 kB 00:00 2024-09-18T07:12:15.862 INFO:teuthology.orchestra.run.smithi070.stdout:(88/129): libarrow-9.0.0-11.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-09-18T07:12:15.882 INFO:teuthology.orchestra.run.smithi172.stdout:(100/129): protobuf-compiler-3.14.0-13.el9.x86_ 874 kB/s | 863 kB 00:00 2024-09-18T07:12:15.887 INFO:teuthology.orchestra.run.smithi070.stdout:(89/129): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-18T07:12:15.916 INFO:teuthology.orchestra.run.smithi172.stdout:(101/129): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-18T07:12:15.929 INFO:teuthology.orchestra.run.smithi070.stdout:(90/129): protobuf-compiler-3.14.0-13.el9.x86_6 915 kB/s | 863 kB 00:00 2024-09-18T07:12:15.950 INFO:teuthology.orchestra.run.smithi172.stdout:(102/129): python3-google-auth-2.34.0-1.el9.noa 2.1 MB/s | 220 kB 00:00 2024-09-18T07:12:15.954 INFO:teuthology.orchestra.run.smithi070.stdout:(91/129): liboath-2.6.7-2.el9.x86_64.rpm 727 kB/s | 49 kB 00:00 2024-09-18T07:12:15.975 INFO:teuthology.orchestra.run.smithi172.stdout:(103/129): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-18T07:12:15.988 INFO:teuthology.orchestra.run.smithi070.stdout:(92/129): luarocks-3.9.2-5.el9.noarch.rpm 5.7 MB/s | 151 kB 00:00 2024-09-18T07:12:16.000 INFO:teuthology.orchestra.run.smithi172.stdout:(104/129): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-18T07:12:16.022 INFO:teuthology.orchestra.run.smithi070.stdout:(93/129): libunwind-1.6.2-1.el9.x86_64.rpm 729 kB/s | 67 kB 00:00 2024-09-18T07:12:16.025 INFO:teuthology.orchestra.run.smithi172.stdout:(105/129): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-09-18T07:12:16.051 INFO:teuthology.orchestra.run.smithi172.stdout:(106/129): python3-jaraco-collections-3.0.0-8.e 458 kB/s | 23 kB 00:00 2024-09-18T07:12:16.064 INFO:teuthology.orchestra.run.smithi070.stdout:(94/129): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-18T07:12:16.076 INFO:teuthology.orchestra.run.smithi172.stdout:(107/129): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-18T07:12:16.102 INFO:teuthology.orchestra.run.smithi172.stdout:(108/129): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-18T07:12:16.114 INFO:teuthology.orchestra.run.smithi070.stdout:(95/129): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 872 kB/s | 43 kB 00:00 2024-09-18T07:12:16.127 INFO:teuthology.orchestra.run.smithi172.stdout:(109/129): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-18T07:12:16.148 INFO:teuthology.orchestra.run.smithi070.stdout:(96/129): python3-cachetools-4.2.4-1.el9.noarch 955 kB/s | 32 kB 00:00 2024-09-18T07:12:16.160 INFO:teuthology.orchestra.run.smithi172.stdout:(110/129): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-18T07:12:16.173 INFO:teuthology.orchestra.run.smithi070.stdout:(97/129): python3-certifi-2023.05.07-4.el9.noar 567 kB/s | 14 kB 00:00 2024-09-18T07:12:16.186 INFO:teuthology.orchestra.run.smithi172.stdout:(111/129): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-18T07:12:16.199 INFO:teuthology.orchestra.run.smithi070.stdout:(98/129): python3-asyncssh-2.13.2-5.el9.noarch. 3.0 MB/s | 548 kB 00:00 2024-09-18T07:12:16.219 INFO:teuthology.orchestra.run.smithi172.stdout:(112/129): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-18T07:12:16.224 INFO:teuthology.orchestra.run.smithi070.stdout:(99/129): python3-cheroot-8.6.0-4.el9.noarch.rp 3.3 MB/s | 172 kB 00:00 2024-09-18T07:12:16.245 INFO:teuthology.orchestra.run.smithi172.stdout:(113/129): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-18T07:12:16.266 INFO:teuthology.orchestra.run.smithi070.stdout:(100/129): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-18T07:12:16.278 INFO:teuthology.orchestra.run.smithi172.stdout:(114/129): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-18T07:12:16.292 INFO:teuthology.orchestra.run.smithi070.stdout:(101/129): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-18T07:12:16.303 INFO:teuthology.orchestra.run.smithi172.stdout:(115/129): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-18T07:12:16.325 INFO:teuthology.orchestra.run.smithi070.stdout:(102/129): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-18T07:12:16.329 INFO:teuthology.orchestra.run.smithi172.stdout:(116/129): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-18T07:12:16.351 INFO:teuthology.orchestra.run.smithi070.stdout:(103/129): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-18T07:12:16.354 INFO:teuthology.orchestra.run.smithi172.stdout:(117/129): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-18T07:12:16.376 INFO:teuthology.orchestra.run.smithi070.stdout:(104/129): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-18T07:12:16.379 INFO:teuthology.orchestra.run.smithi172.stdout:(118/129): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-18T07:12:16.396 INFO:teuthology.orchestra.run.smithi172.stdout:(119/129): python3-tempora-5.0.0-2.el9.noarch.r 857 kB/s | 36 kB 00:00 2024-09-18T07:12:16.401 INFO:teuthology.orchestra.run.smithi070.stdout:(105/129): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-18T07:12:16.426 INFO:teuthology.orchestra.run.smithi070.stdout:(106/129): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-09-18T07:12:16.438 INFO:teuthology.orchestra.run.smithi172.stdout:(120/129): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-09-18T07:12:16.452 INFO:teuthology.orchestra.run.smithi070.stdout:(107/129): python3-jaraco-text-3.2.0-6.el9.noar 787 kB/s | 20 kB 00:00 2024-09-18T07:12:16.464 INFO:teuthology.orchestra.run.smithi172.stdout:(121/129): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-18T07:12:16.489 INFO:teuthology.orchestra.run.smithi172.stdout:(122/129): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-18T07:12:16.510 INFO:teuthology.orchestra.run.smithi070.stdout:(108/129): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-18T07:12:16.514 INFO:teuthology.orchestra.run.smithi172.stdout:(123/129): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-18T07:12:16.536 INFO:teuthology.orchestra.run.smithi070.stdout:(109/129): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-18T07:12:16.539 INFO:teuthology.orchestra.run.smithi172.stdout:(124/129): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-18T07:12:16.561 INFO:teuthology.orchestra.run.smithi070.stdout:(110/129): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-18T07:12:16.565 INFO:teuthology.orchestra.run.smithi172.stdout:(125/129): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-18T07:12:16.586 INFO:teuthology.orchestra.run.smithi070.stdout:(111/129): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-18T07:12:16.590 INFO:teuthology.orchestra.run.smithi172.stdout:(126/129): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-18T07:12:16.629 INFO:teuthology.orchestra.run.smithi070.stdout:(112/129): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-09-18T07:12:16.654 INFO:teuthology.orchestra.run.smithi070.stdout:(113/129): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-18T07:12:16.665 INFO:teuthology.orchestra.run.smithi172.stdout:(127/129): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-18T07:12:16.679 INFO:teuthology.orchestra.run.smithi070.stdout:(114/129): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-18T07:12:16.704 INFO:teuthology.orchestra.run.smithi070.stdout:(115/129): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-18T07:12:16.730 INFO:teuthology.orchestra.run.smithi070.stdout:(116/129): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-18T07:12:16.755 INFO:teuthology.orchestra.run.smithi070.stdout:(117/129): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-18T07:12:16.780 INFO:teuthology.orchestra.run.smithi070.stdout:(118/129): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-18T07:12:16.805 INFO:teuthology.orchestra.run.smithi070.stdout:(119/129): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-18T07:12:16.831 INFO:teuthology.orchestra.run.smithi070.stdout:(120/129): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-18T07:12:16.856 INFO:teuthology.orchestra.run.smithi070.stdout:(121/129): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-18T07:12:16.881 INFO:teuthology.orchestra.run.smithi070.stdout:(122/129): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-18T07:12:16.907 INFO:teuthology.orchestra.run.smithi070.stdout:(123/129): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-18T07:12:16.907 INFO:teuthology.orchestra.run.smithi172.stdout:(128/129): librados2-19.3.0-4953.g82e3d596.el9. 10 MB/s | 3.5 MB 00:00 2024-09-18T07:12:16.932 INFO:teuthology.orchestra.run.smithi070.stdout:(124/129): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-09-18T07:12:16.957 INFO:teuthology.orchestra.run.smithi070.stdout:(125/129): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-18T07:12:16.999 INFO:teuthology.orchestra.run.smithi172.stdout:(129/129): librbd1-19.3.0-4953.g82e3d596.el9.x8 8.0 MB/s | 3.2 MB 00:00 2024-09-18T07:12:17.005 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-18T07:12:17.005 INFO:teuthology.orchestra.run.smithi172.stdout:Total 24 MB/s | 212 MB 00:08 2024-09-18T07:12:17.033 INFO:teuthology.orchestra.run.smithi070.stdout:(126/129): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-18T07:12:17.233 INFO:teuthology.orchestra.run.smithi070.stdout:(127/129): librados2-19.3.0-4953.g82e3d596.el9. 13 MB/s | 3.5 MB 00:00 2024-09-18T07:12:17.433 INFO:teuthology.orchestra.run.smithi070.stdout:(128/129): librbd1-19.3.0-4953.g82e3d596.el9.x8 8.1 MB/s | 3.2 MB 00:00 2024-09-18T07:12:17.855 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-18T07:12:17.963 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-18T07:12:17.963 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-18T07:12:19.483 INFO:teuthology.orchestra.run.smithi070.stdout:(129/129): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.2 MB/s | 16 MB 00:05 2024-09-18T07:12:19.489 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-18T07:12:19.489 INFO:teuthology.orchestra.run.smithi070.stdout:Total 13 MB/s | 212 MB 00:15 2024-09-18T07:12:19.947 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-18T07:12:19.947 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-18T07:12:20.315 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-18T07:12:20.419 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-18T07:12:20.419 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-18T07:12:22.459 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-18T07:12:22.459 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-18T07:12:25.083 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-18T07:12:25.159 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/131 2024-09-18T07:12:25.193 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/131 2024-09-18T07:12:25.264 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/131 2024-09-18T07:12:25.598 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/131 2024-09-18T07:12:25.735 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librados2-2:19.3.0-4953.g82e3d596.el9.x86_64 5/131 2024-09-18T07:12:25.858 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:19.3.0-4953.g82e3d596.el9.x86_64 5/131 2024-09-18T07:12:25.895 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs2-2:19.3.0-4953.g82e3d596.el9.x86_64 6/131 2024-09-18T07:12:25.986 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs2-2:19.3.0-4953.g82e3d596.el9.x86_64 6/131 2024-09-18T07:12:26.037 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rados-2:19.3.0-4953.g82e3d596.el9.x86_64 7/131 2024-09-18T07:12:26.066 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/131 2024-09-18T07:12:26.097 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/131 2024-09-18T07:12:26.145 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 10/131 2024-09-18T07:12:26.173 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephsqlite-2:19.3.0-4953.g82e3d596.el9.x86_64 11/131 2024-09-18T07:12:26.277 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4953.g82e3d596.el9.x86_64 11/131 2024-09-18T07:12:26.307 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libradosstriper1-2:19.3.0-4953.g82e3d596.el9.x86 12/131 2024-09-18T07:12:26.396 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4953.g82e3d596.el9.x86 12/131 2024-09-18T07:12:26.425 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 13/131 2024-09-18T07:12:26.452 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 13/131 2024-09-18T07:12:26.837 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 14/131 2024-09-18T07:12:26.875 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 14/131 2024-09-18T07:12:26.983 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/131 2024-09-18T07:12:27.024 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/131 2024-09-18T07:12:27.067 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : liboath-2.6.7-2.el9.x86_64 17/131 2024-09-18T07:12:27.141 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 18/131 2024-09-18T07:12:27.194 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 19/131 2024-09-18T07:12:27.234 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-packaging-20.9-5.el9.noarch 20/131 2024-09-18T07:12:27.277 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/131 2024-09-18T07:12:27.309 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 22/131 2024-09-18T07:12:27.337 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-5.4.4-4.el9.x86_64 23/131 2024-09-18T07:12:27.410 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 24/131 2024-09-18T07:12:27.478 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 25/131 2024-09-18T07:12:27.524 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 26/131 2024-09-18T07:12:27.567 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 27/131 2024-09-18T07:12:27.646 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 28/131 2024-09-18T07:12:27.678 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-common-2:19.3.0-4953.g82e3d596.el9. 29/131 2024-09-18T07:12:27.721 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-argparse-2:19.3.0-4953.g82e3d596.el 30/131 2024-09-18T07:12:27.749 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cephfs-2:19.3.0-4953.g82e3d596.el9.x86_6 31/131 2024-09-18T07:12:27.822 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 32/131 2024-09-18T07:12:27.822 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-18T07:12:27.865 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : luarocks-3.9.2-5.el9.noarch 33/131 2024-09-18T07:12:27.896 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/131 2024-09-18T07:12:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/131 2024-09-18T07:12:27.949 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 34/131 2024-09-18T07:12:27.980 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/131 2024-09-18T07:12:28.094 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 35/131 2024-09-18T07:12:28.151 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 36/131 2024-09-18T07:12:28.335 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/131 2024-09-18T07:12:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:19.3.0-4953.g82e3d596.el9.x86_64 5/131 2024-09-18T07:12:28.577 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:19.3.0-4953.g82e3d596.el9.x86_64 5/131 2024-09-18T07:12:28.599 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:19.3.0-4953.g82e3d596.el9.x86_64 6/131 2024-09-18T07:12:28.690 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:19.3.0-4953.g82e3d596.el9.x86_64 6/131 2024-09-18T07:12:28.697 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rsa-4.9-2.el9.noarch 37/131 2024-09-18T07:12:28.741 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:19.3.0-4953.g82e3d596.el9.x86_64 7/131 2024-09-18T07:12:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/131 2024-09-18T07:12:28.788 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 38/131 2024-09-18T07:12:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/131 2024-09-18T07:12:28.816 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 39/131 2024-09-18T07:12:28.859 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 40/131 2024-09-18T07:12:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 10/131 2024-09-18T07:12:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:19.3.0-4953.g82e3d596.el9.x86_64 11/131 2024-09-18T07:12:28.897 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librados-devel-2:19.3.0-4953.g82e3d596.el9.x86_6 41/131 2024-09-18T07:12:28.930 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 42/131 2024-09-18T07:12:28.990 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4953.g82e3d596.el9.x86_64 11/131 2024-09-18T07:12:29.020 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:19.3.0-4953.g82e3d596.el9.x86 12/131 2024-09-18T07:12:29.023 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 43/131 2024-09-18T07:12:29.079 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 44/131 2024-09-18T07:12:29.083 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4953.g82e3d596.el9.x86 12/131 2024-09-18T07:12:29.112 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 13/131 2024-09-18T07:12:29.138 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 13/131 2024-09-18T07:12:29.161 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/131 2024-09-18T07:12:29.207 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 46/131 2024-09-18T07:12:29.243 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 47/131 2024-09-18T07:12:29.290 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/131 2024-09-18T07:12:29.332 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 14/131 2024-09-18T07:12:29.336 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/131 2024-09-18T07:12:29.363 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 14/131 2024-09-18T07:12:29.377 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/131 2024-09-18T07:12:29.473 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/131 2024-09-18T07:12:29.503 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/131 2024-09-18T07:12:29.516 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/131 2024-09-18T07:12:29.538 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.7-2.el9.x86_64 17/131 2024-09-18T07:12:29.562 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/131 2024-09-18T07:12:29.612 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 53/131 2024-09-18T07:12:29.618 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 18/131 2024-09-18T07:12:29.672 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 19/131 2024-09-18T07:12:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-packaging-20.9-5.el9.noarch 20/131 2024-09-18T07:12:29.717 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/131 2024-09-18T07:12:29.754 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 55/131 2024-09-18T07:12:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/131 2024-09-18T07:12:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 22/131 2024-09-18T07:12:29.794 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/131 2024-09-18T07:12:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-5.4.4-4.el9.x86_64 23/131 2024-09-18T07:12:29.827 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 57/131 2024-09-18T07:12:29.852 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 58/131 2024-09-18T07:12:29.887 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 24/131 2024-09-18T07:12:29.940 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 25/131 2024-09-18T07:12:29.949 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 59/131 2024-09-18T07:12:29.975 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 26/131 2024-09-18T07:12:30.014 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 27/131 2024-09-18T07:12:30.080 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 60/131 2024-09-18T07:12:30.102 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 28/131 2024-09-18T07:12:30.120 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 61/131 2024-09-18T07:12:30.133 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:19.3.0-4953.g82e3d596.el9. 29/131 2024-09-18T07:12:30.165 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 62/131 2024-09-18T07:12:30.177 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:19.3.0-4953.g82e3d596.el 30/131 2024-09-18T07:12:30.205 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:19.3.0-4953.g82e3d596.el9.x86_6 31/131 2024-09-18T07:12:30.207 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 63/131 2024-09-18T07:12:30.242 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 64/131 2024-09-18T07:12:30.271 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 65/131 2024-09-18T07:12:30.277 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 32/131 2024-09-18T07:12:30.306 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 66/131 2024-09-18T07:12:30.320 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : luarocks-3.9.2-5.el9.noarch 33/131 2024-09-18T07:12:30.339 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 67/131 2024-09-18T07:12:30.382 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 68/131 2024-09-18T07:12:30.396 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 34/131 2024-09-18T07:12:30.415 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 69/131 2024-09-18T07:12:30.504 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 70/131 2024-09-18T07:12:30.532 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 35/131 2024-09-18T07:12:30.563 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 71/131 2024-09-18T07:12:30.581 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 36/131 2024-09-18T07:12:30.594 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/131 2024-09-18T07:12:30.701 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 73/131 2024-09-18T07:12:30.753 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 74/131 2024-09-18T07:12:31.131 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el9.noarch 37/131 2024-09-18T07:12:31.210 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 38/131 2024-09-18T07:12:31.239 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 39/131 2024-09-18T07:12:31.282 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 40/131 2024-09-18T07:12:31.320 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:19.3.0-4953.g82e3d596.el9.x86_6 41/131 2024-09-18T07:12:31.361 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 42/131 2024-09-18T07:12:31.462 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 43/131 2024-09-18T07:12:31.499 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 75/131 2024-09-18T07:12:31.543 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 44/131 2024-09-18T07:12:31.550 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 76/131 2024-09-18T07:12:31.600 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 77/131 2024-09-18T07:12:31.615 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/131 2024-09-18T07:12:31.647 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 46/131 2024-09-18T07:12:31.683 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 47/131 2024-09-18T07:12:31.710 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 78/131 2024-09-18T07:12:31.751 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 79/131 2024-09-18T07:12:31.763 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/131 2024-09-18T07:12:31.809 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/131 2024-09-18T07:12:31.850 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/131 2024-09-18T07:12:31.936 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-0.3.26-2.el9.x86_64 80/131 2024-09-18T07:12:31.973 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 81/131 2024-09-18T07:12:31.997 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/131 2024-09-18T07:12:32.044 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/131 2024-09-18T07:12:32.054 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 82/131 2024-09-18T07:12:32.085 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 53/131 2024-09-18T07:12:32.189 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/131 2024-09-18T07:12:32.220 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 55/131 2024-09-18T07:12:32.260 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/131 2024-09-18T07:12:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 57/131 2024-09-18T07:12:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 58/131 2024-09-18T07:12:32.425 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 59/131 2024-09-18T07:12:32.553 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 60/131 2024-09-18T07:12:32.602 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 61/131 2024-09-18T07:12:32.640 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 62/131 2024-09-18T07:12:32.682 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 63/131 2024-09-18T07:12:32.717 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 64/131 2024-09-18T07:12:32.732 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 83/131 2024-09-18T07:12:32.746 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 65/131 2024-09-18T07:12:32.781 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 66/131 2024-09-18T07:12:32.814 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 67/131 2024-09-18T07:12:32.852 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 84/131 2024-09-18T07:12:32.856 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 68/131 2024-09-18T07:12:32.899 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 69/131 2024-09-18T07:12:32.995 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 70/131 2024-09-18T07:12:33.265 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 71/131 2024-09-18T07:12:33.303 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/131 2024-09-18T07:12:33.415 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 73/131 2024-09-18T07:12:33.480 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 74/131 2024-09-18T07:12:34.197 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/131 2024-09-18T07:12:34.215 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 75/131 2024-09-18T07:12:34.271 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 76/131 2024-09-18T07:12:34.291 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 86/131 2024-09-18T07:12:34.304 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 77/131 2024-09-18T07:12:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 78/131 2024-09-18T07:12:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 79/131 2024-09-18T07:12:34.630 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.26-2.el9.x86_64 80/131 2024-09-18T07:12:34.631 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 87/131 2024-09-18T07:12:34.661 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 81/131 2024-09-18T07:12:34.677 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librbd1-2:19.3.0-4953.g82e3d596.el9.x86_64 88/131 2024-09-18T07:12:34.738 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 82/131 2024-09-18T07:12:34.777 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:19.3.0-4953.g82e3d596.el9.x86_64 88/131 2024-09-18T07:12:34.807 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rbd-2:19.3.0-4953.g82e3d596.el9.x86_64 89/131 2024-09-18T07:12:34.868 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 90/131 2024-09-18T07:12:35.360 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 91/131 2024-09-18T07:12:35.388 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librgw2-2:19.3.0-4953.g82e3d596.el9.x86_64 92/131 2024-09-18T07:12:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 83/131 2024-09-18T07:12:35.451 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librgw2-2:19.3.0-4953.g82e3d596.el9.x86_64 92/131 2024-09-18T07:12:35.473 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rgw-2:19.3.0-4953.g82e3d596.el9.x86_64 93/131 2024-09-18T07:12:35.519 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 84/131 2024-09-18T07:12:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/131 2024-09-18T07:12:36.879 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 86/131 2024-09-18T07:12:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 87/131 2024-09-18T07:12:37.250 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:19.3.0-4953.g82e3d596.el9.x86_64 88/131 2024-09-18T07:12:37.340 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:19.3.0-4953.g82e3d596.el9.x86_64 88/131 2024-09-18T07:12:37.371 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:19.3.0-4953.g82e3d596.el9.x86_64 89/131 2024-09-18T07:12:37.407 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 90/131 2024-09-18T07:12:37.882 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.3.0-4953.g82e3d596.el9.x86_64 94/131 2024-09-18T07:12:37.900 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 91/131 2024-09-18T07:12:37.928 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-common-2:19.3.0-4953.g82e3d596.el9.x86_64 94/131 2024-09-18T07:12:37.936 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:19.3.0-4953.g82e3d596.el9.x86_64 92/131 2024-09-18T07:12:38.008 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:19.3.0-4953.g82e3d596.el9.x86_64 92/131 2024-09-18T07:12:38.046 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:19.3.0-4953.g82e3d596.el9.x86_64 93/131 2024-09-18T07:12:38.492 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.3.0-4953.g82e3d596.el9.x86_64 94/131 2024-09-18T07:12:38.531 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-base-2:19.3.0-4953.g82e3d596.el9.x86_64 95/131 2024-09-18T07:12:38.624 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-base-2:19.3.0-4953.g82e3d596.el9.x86_64 95/131 2024-09-18T07:12:38.624 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-18T07:12:38.624 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-18T07:12:38.624 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:12:38.657 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-selinux-2:19.3.0-4953.g82e3d596.el9.x86_64 96/131 2024-09-18T07:12:40.386 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.3.0-4953.g82e3d596.el9.x86_64 94/131 2024-09-18T07:12:40.426 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:19.3.0-4953.g82e3d596.el9.x86_64 94/131 2024-09-18T07:12:40.989 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.3.0-4953.g82e3d596.el9.x86_64 94/131 2024-09-18T07:12:41.038 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:19.3.0-4953.g82e3d596.el9.x86_64 95/131 2024-09-18T07:12:41.129 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:19.3.0-4953.g82e3d596.el9.x86_64 95/131 2024-09-18T07:12:41.129 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-18T07:12:41.129 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-18T07:12:41.130 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:12:41.164 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:19.3.0-4953.g82e3d596.el9.x86_64 96/131 2024-09-18T07:12:50.439 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4953.g82e3d596.el9.x86_64 96/131 2024-09-18T07:12:50.440 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /sys 2024-09-18T07:12:50.440 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /proc 2024-09-18T07:12:50.440 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /mnt 2024-09-18T07:12:50.440 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /var/tmp 2024-09-18T07:12:50.440 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /home 2024-09-18T07:12:50.440 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /root 2024-09-18T07:12:50.440 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /tmp 2024-09-18T07:12:50.440 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:12:50.618 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-volume-2:19.3.0-4953.g82e3d596.el9.noarch 97/131 2024-09-18T07:12:50.634 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-volume-2:19.3.0-4953.g82e3d596.el9.noarch 97/131 2024-09-18T07:12:50.634 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:12:50.634 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-18T07:12:50.634 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:12:52.196 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-osd-2:19.3.0-4953.g82e3d596.el9.x86_64 98/131 2024-09-18T07:12:52.225 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-osd-2:19.3.0-4953.g82e3d596.el9.x86_64 98/131 2024-09-18T07:12:52.225 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:12:52.225 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-18T07:12:52.225 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T07:12:52.225 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T07:12:52.225 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:12:52.472 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mds-2:19.3.0-4953.g82e3d596.el9.x86_64 99/131 2024-09-18T07:12:52.496 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mds-2:19.3.0-4953.g82e3d596.el9.x86_64 99/131 2024-09-18T07:12:52.496 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:12:52.496 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-18T07:12:52.496 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T07:12:52.496 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T07:12:52.496 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:12:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4953.g82e3d596.el9.x86_64 96/131 2024-09-18T07:12:52.895 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2024-09-18T07:12:52.895 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2024-09-18T07:12:52.895 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2024-09-18T07:12:52.895 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2024-09-18T07:12:52.895 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2024-09-18T07:12:52.895 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /root 2024-09-18T07:12:52.896 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2024-09-18T07:12:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:12:52.948 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mon-2:19.3.0-4953.g82e3d596.el9.x86_64 100/131 2024-09-18T07:12:52.972 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mon-2:19.3.0-4953.g82e3d596.el9.x86_64 100/131 2024-09-18T07:12:52.972 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:12:52.972 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-18T07:12:52.973 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T07:12:52.973 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T07:12:52.973 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:12:53.043 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ply-3.11-14.el9.noarch 101/131 2024-09-18T07:12:53.070 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-volume-2:19.3.0-4953.g82e3d596.el9.noarch 97/131 2024-09-18T07:12:53.085 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-volume-2:19.3.0-4953.g82e3d596.el9.noarch 97/131 2024-09-18T07:12:53.086 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:12:53.086 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-18T07:12:53.086 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:12:53.098 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 102/131 2024-09-18T07:12:53.287 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 103/131 2024-09-18T07:12:53.364 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 104/131 2024-09-18T07:12:53.442 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 105/131 2024-09-18T07:12:53.539 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 106/131 2024-09-18T07:12:53.680 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 107/131 2024-09-18T07:12:53.727 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 108/131 2024-09-18T07:12:53.771 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 109/131 2024-09-18T07:12:53.798 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : fuse-2.9.9-16.el9.x86_64 110/131 2024-09-18T07:12:54.254 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:19.3.0-4953.g82e3d596.el9.noarch 111/131 2024-09-18T07:12:54.280 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephadm-2:19.3.0-4953.g82e3d596.el9.noarch 111/131 2024-09-18T07:12:54.312 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4953.g82e3d596.e 112/131 2024-09-18T07:12:54.378 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4953.g82e3d596. 113/131 2024-09-18T07:12:54.411 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4953.g82e3d596.el9.noa 114/131 2024-09-18T07:12:54.556 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:19.3.0-4953.g82e3d596.el9.x86_64 98/131 2024-09-18T07:12:54.583 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:19.3.0-4953.g82e3d596.el9.x86_64 98/131 2024-09-18T07:12:54.583 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:12:54.583 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-18T07:12:54.583 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T07:12:54.583 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T07:12:54.583 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:12:54.815 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:19.3.0-4953.g82e3d596.el9.x86_64 99/131 2024-09-18T07:12:54.838 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:19.3.0-4953.g82e3d596.el9.x86_64 99/131 2024-09-18T07:12:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:12:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-18T07:12:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T07:12:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T07:12:54.839 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:12:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:19.3.0-4953.g82e3d596.el9.x86_64 100/131 2024-09-18T07:12:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:19.3.0-4953.g82e3d596.el9.x86_64 100/131 2024-09-18T07:12:55.306 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:12:55.306 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-18T07:12:55.306 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T07:12:55.307 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T07:12:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:12:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ply-3.11-14.el9.noarch 101/131 2024-09-18T07:12:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 102/131 2024-09-18T07:12:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 103/131 2024-09-18T07:12:55.691 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 104/131 2024-09-18T07:12:55.759 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 105/131 2024-09-18T07:12:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 106/131 2024-09-18T07:12:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 107/131 2024-09-18T07:12:56.033 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 108/131 2024-09-18T07:12:56.065 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 109/131 2024-09-18T07:12:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.9-16.el9.x86_64 110/131 2024-09-18T07:12:56.523 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:19.3.0-4953.g82e3d596.el9.noarch 111/131 2024-09-18T07:12:56.558 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:19.3.0-4953.g82e3d596.el9.noarch 111/131 2024-09-18T07:12:56.590 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4953.g82e3d596.e 112/131 2024-09-18T07:12:56.664 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4953.g82e3d596. 113/131 2024-09-18T07:12:56.689 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4953.g82e3d596.el9.noa 114/131 2024-09-18T07:12:57.616 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4953.g82e3d596.el9.noa 114/131 2024-09-18T07:12:57.676 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4953.g82e3d596.el9.n 115/131 2024-09-18T07:12:58.638 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4953.g82e3d596.el9.n 115/131 2024-09-18T07:12:58.668 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4953.g82e 116/131 2024-09-18T07:12:58.784 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4953.g82e 116/131 2024-09-18T07:12:58.946 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4953.g82e3d596.el 117/131 2024-09-18T07:12:58.978 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-2:19.3.0-4953.g82e3d596.el9.x86_64 118/131 2024-09-18T07:12:59.002 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4953.g82e3d596.el9.x86_64 118/131 2024-09-18T07:12:59.002 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:12:59.003 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-18T07:12:59.003 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T07:12:59.003 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T07:12:59.003 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:12:59.054 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-rook-2:19.3.0-4953.g82e3d596.el9.noarch 119/131 2024-09-18T07:12:59.066 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4953.g82e3d596.el9.noarch 119/131 2024-09-18T07:12:59.164 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-2:19.3.0-4953.g82e3d596.el9.x86_64 120/131 2024-09-18T07:12:59.853 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4953.g82e3d596.el9.noa 114/131 2024-09-18T07:12:59.912 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4953.g82e3d596.el9.n 115/131 2024-09-18T07:13:00.114 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-fuse-2:19.3.0-4953.g82e3d596.el9.x86_64 121/131 2024-09-18T07:13:00.155 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-radosgw-2:19.3.0-4953.g82e3d596.el9.x86_64 122/131 2024-09-18T07:13:00.179 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4953.g82e3d596.el9.x86_64 122/131 2024-09-18T07:13:00.179 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:13:00.180 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-18T07:13:00.180 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T07:13:00.180 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T07:13:00.180 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:00.223 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4953.g82e3d 123/131 2024-09-18T07:13:00.246 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4953.g82e3d 123/131 2024-09-18T07:13:00.247 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:13:00.247 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-18T07:13:00.247 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:00.565 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-mirror-2:19.3.0-4953.g82e3d596.el9.x86_64 124/131 2024-09-18T07:13:00.588 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4953.g82e3d596.el9.x86_64 124/131 2024-09-18T07:13:00.589 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:13:00.589 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-18T07:13:00.589 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T07:13:00.589 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T07:13:00.589 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4953.g82e3d596.el9.n 115/131 2024-09-18T07:13:00.904 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4953.g82e 116/131 2024-09-18T07:13:01.020 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4953.g82e 116/131 2024-09-18T07:13:01.174 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4953.g82e3d596.el 117/131 2024-09-18T07:13:01.206 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:19.3.0-4953.g82e3d596.el9.x86_64 118/131 2024-09-18T07:13:01.230 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4953.g82e3d596.el9.x86_64 118/131 2024-09-18T07:13:01.230 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:13:01.230 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-18T07:13:01.230 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T07:13:01.230 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T07:13:01.230 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:01.273 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:19.3.0-4953.g82e3d596.el9.noarch 119/131 2024-09-18T07:13:01.284 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4953.g82e3d596.el9.noarch 119/131 2024-09-18T07:13:01.382 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:19.3.0-4953.g82e3d596.el9.x86_64 120/131 2024-09-18T07:13:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:19.3.0-4953.g82e3d596.el9.x86_64 121/131 2024-09-18T07:13:02.359 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:19.3.0-4953.g82e3d596.el9.x86_64 122/131 2024-09-18T07:13:02.382 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4953.g82e3d596.el9.x86_64 122/131 2024-09-18T07:13:02.382 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:13:02.382 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-18T07:13:02.382 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T07:13:02.383 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T07:13:02.383 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:02.434 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4953.g82e3d 123/131 2024-09-18T07:13:02.460 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4953.g82e3d 123/131 2024-09-18T07:13:02.460 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:13:02.461 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-18T07:13:02.461 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:02.777 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:19.3.0-4953.g82e3d596.el9.x86_64 124/131 2024-09-18T07:13:02.800 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4953.g82e3d596.el9.x86_64 124/131 2024-09-18T07:13:02.800 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T07:13:02.800 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-18T07:13:02.800 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T07:13:02.800 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T07:13:02.800 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:04.935 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-test-2:19.3.0-4953.g82e3d596.el9.x86_64 125/131 2024-09-18T07:13:04.976 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-fuse-2:19.3.0-4953.g82e3d596.el9.x86_64 126/131 2024-09-18T07:13:05.007 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-nbd-2:19.3.0-4953.g82e3d596.el9.x86_64 127/131 2024-09-18T07:13:05.050 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : sqlite-devel-3.34.1-7.el9.x86_64 128/131 2024-09-18T07:13:05.081 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs-devel-2:19.3.0-4953.g82e3d596.el9.x86_ 129/131 2024-09-18T07:13:05.081 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 130/131 2024-09-18T07:13:05.189 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 130/131 2024-09-18T07:13:05.189 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 131/131 2024-09-18T07:13:07.147 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:19.3.0-4953.g82e3d596.el9.x86_64 125/131 2024-09-18T07:13:07.188 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:19.3.0-4953.g82e3d596.el9.x86_64 126/131 2024-09-18T07:13:07.219 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:19.3.0-4953.g82e3d596.el9.x86_64 127/131 2024-09-18T07:13:07.245 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : sqlite-devel-3.34.1-7.el9.x86_64 128/131 2024-09-18T07:13:07.276 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:19.3.0-4953.g82e3d596.el9.x86_ 129/131 2024-09-18T07:13:07.276 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 130/131 2024-09-18T07:13:07.343 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 130/131 2024-09-18T07:13:07.343 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 131/131 2024-09-18T07:13:11.151 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 131/131 2024-09-18T07:13:11.151 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-2:19.3.0-4953.g82e3d596.el9.x86_64 1/131 2024-09-18T07:13:11.151 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-base-2:19.3.0-4953.g82e3d596.el9.x86_64 2/131 2024-09-18T07:13:11.151 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-common-2:19.3.0-4953.g82e3d596.el9.x86_64 3/131 2024-09-18T07:13:11.151 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-fuse-2:19.3.0-4953.g82e3d596.el9.x86_64 4/131 2024-09-18T07:13:11.151 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4953.g82e3d 5/131 2024-09-18T07:13:11.152 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mds-2:19.3.0-4953.g82e3d596.el9.x86_64 6/131 2024-09-18T07:13:11.152 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-2:19.3.0-4953.g82e3d596.el9.x86_64 7/131 2024-09-18T07:13:11.152 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mon-2:19.3.0-4953.g82e3d596.el9.x86_64 8/131 2024-09-18T07:13:11.152 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-osd-2:19.3.0-4953.g82e3d596.el9.x86_64 9/131 2024-09-18T07:13:11.152 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-radosgw-2:19.3.0-4953.g82e3d596.el9.x86_64 10/131 2024-09-18T07:13:11.152 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-selinux-2:19.3.0-4953.g82e3d596.el9.x86_64 11/131 2024-09-18T07:13:11.152 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-test-2:19.3.0-4953.g82e3d596.el9.x86_64 12/131 2024-09-18T07:13:11.152 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs-devel-2:19.3.0-4953.g82e3d596.el9.x86_ 13/131 2024-09-18T07:13:11.152 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs2-2:19.3.0-4953.g82e3d596.el9.x86_64 14/131 2024-09-18T07:13:11.152 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephsqlite-2:19.3.0-4953.g82e3d596.el9.x86_64 15/131 2024-09-18T07:13:11.153 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados-devel-2:19.3.0-4953.g82e3d596.el9.x86_6 16/131 2024-09-18T07:13:11.153 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libradosstriper1-2:19.3.0-4953.g82e3d596.el9.x86 17/131 2024-09-18T07:13:11.153 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librgw2-2:19.3.0-4953.g82e3d596.el9.x86_64 18/131 2024-09-18T07:13:11.153 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4953.g82e3d596.el 19/131 2024-09-18T07:13:11.153 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-common-2:19.3.0-4953.g82e3d596.el9. 20/131 2024-09-18T07:13:11.153 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cephfs-2:19.3.0-4953.g82e3d596.el9.x86_6 21/131 2024-09-18T07:13:11.153 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rados-2:19.3.0-4953.g82e3d596.el9.x86_64 22/131 2024-09-18T07:13:11.153 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rbd-2:19.3.0-4953.g82e3d596.el9.x86_64 23/131 2024-09-18T07:13:11.153 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rgw-2:19.3.0-4953.g82e3d596.el9.x86_64 24/131 2024-09-18T07:13:11.153 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-fuse-2:19.3.0-4953.g82e3d596.el9.x86_64 25/131 2024-09-18T07:13:11.153 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-mirror-2:19.3.0-4953.g82e3d596.el9.x86_64 26/131 2024-09-18T07:13:11.154 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-nbd-2:19.3.0-4953.g82e3d596.el9.x86_64 27/131 2024-09-18T07:13:11.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4953.g82e3d596. 28/131 2024-09-18T07:13:11.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4953.g82e3d596.el9.noa 29/131 2024-09-18T07:13:11.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4953.g82e3d596.el9.n 30/131 2024-09-18T07:13:11.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4953.g82e 31/131 2024-09-18T07:13:11.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4953.g82e3d596.el 32/131 2024-09-18T07:13:11.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4953.g82e3d596.el9.noarch 33/131 2024-09-18T07:13:11.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4953.g82e3d596.e 34/131 2024-09-18T07:13:11.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-volume-2:19.3.0-4953.g82e3d596.el9.noarch 35/131 2024-09-18T07:13:11.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephadm-2:19.3.0-4953.g82e3d596.el9.noarch 36/131 2024-09-18T07:13:11.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 37/131 2024-09-18T07:13:11.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 38/131 2024-09-18T07:13:11.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 39/131 2024-09-18T07:13:11.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/131 2024-09-18T07:13:11.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/131 2024-09-18T07:13:11.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/131 2024-09-18T07:13:11.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/131 2024-09-18T07:13:11.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/131 2024-09-18T07:13:11.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 45/131 2024-09-18T07:13:11.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 46/131 2024-09-18T07:13:11.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 47/131 2024-09-18T07:13:11.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 48/131 2024-09-18T07:13:11.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 49/131 2024-09-18T07:13:11.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 50/131 2024-09-18T07:13:11.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 51/131 2024-09-18T07:13:11.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 52/131 2024-09-18T07:13:11.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 53/131 2024-09-18T07:13:11.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 54/131 2024-09-18T07:13:11.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-5.4.4-4.el9.x86_64 55/131 2024-09-18T07:13:11.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 56/131 2024-09-18T07:13:11.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 57/131 2024-09-18T07:13:11.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/131 2024-09-18T07:13:11.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 59/131 2024-09-18T07:13:11.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 60/131 2024-09-18T07:13:11.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 61/131 2024-09-18T07:13:11.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 62/131 2024-09-18T07:13:11.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 63/131 2024-09-18T07:13:11.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 64/131 2024-09-18T07:13:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 65/131 2024-09-18T07:13:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 66/131 2024-09-18T07:13:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 67/131 2024-09-18T07:13:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 68/131 2024-09-18T07:13:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 69/131 2024-09-18T07:13:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 70/131 2024-09-18T07:13:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 71/131 2024-09-18T07:13:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 72/131 2024-09-18T07:13:11.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 73/131 2024-09-18T07:13:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 74/131 2024-09-18T07:13:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 75/131 2024-09-18T07:13:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 76/131 2024-09-18T07:13:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 77/131 2024-09-18T07:13:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 78/131 2024-09-18T07:13:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 79/131 2024-09-18T07:13:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 80/131 2024-09-18T07:13:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 81/131 2024-09-18T07:13:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 82/131 2024-09-18T07:13:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : sqlite-devel-3.34.1-7.el9.x86_64 83/131 2024-09-18T07:13:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/131 2024-09-18T07:13:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 85/131 2024-09-18T07:13:11.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 86/131 2024-09-18T07:13:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 87/131 2024-09-18T07:13:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 88/131 2024-09-18T07:13:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 89/131 2024-09-18T07:13:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 90/131 2024-09-18T07:13:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 91/131 2024-09-18T07:13:11.170 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 92/131 2024-09-18T07:13:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 93/131 2024-09-18T07:13:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 94/131 2024-09-18T07:13:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 95/131 2024-09-18T07:13:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 96/131 2024-09-18T07:13:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 97/131 2024-09-18T07:13:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 98/131 2024-09-18T07:13:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 99/131 2024-09-18T07:13:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 100/131 2024-09-18T07:13:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 101/131 2024-09-18T07:13:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 102/131 2024-09-18T07:13:11.171 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 103/131 2024-09-18T07:13:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 104/131 2024-09-18T07:13:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 105/131 2024-09-18T07:13:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 106/131 2024-09-18T07:13:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 107/131 2024-09-18T07:13:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 108/131 2024-09-18T07:13:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 109/131 2024-09-18T07:13:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 110/131 2024-09-18T07:13:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 111/131 2024-09-18T07:13:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 112/131 2024-09-18T07:13:11.172 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 113/131 2024-09-18T07:13:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 114/131 2024-09-18T07:13:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 115/131 2024-09-18T07:13:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 116/131 2024-09-18T07:13:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 117/131 2024-09-18T07:13:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 118/131 2024-09-18T07:13:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 119/131 2024-09-18T07:13:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 120/131 2024-09-18T07:13:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 121/131 2024-09-18T07:13:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 122/131 2024-09-18T07:13:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 123/131 2024-09-18T07:13:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 124/131 2024-09-18T07:13:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 125/131 2024-09-18T07:13:11.173 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : re2-1:20211101-20.el9.x86_64 126/131 2024-09-18T07:13:11.174 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 127/131 2024-09-18T07:13:11.174 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:19.3.0-4953.g82e3d596.el9.x86_64 128/131 2024-09-18T07:13:11.174 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 129/131 2024-09-18T07:13:11.174 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:19.3.0-4953.g82e3d596.el9.x86_64 130/131 2024-09-18T07:13:13.329 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 131/131 2024-09-18T07:13:13.329 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:19.3.0-4953.g82e3d596.el9.x86_64 1/131 2024-09-18T07:13:13.329 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:19.3.0-4953.g82e3d596.el9.x86_64 2/131 2024-09-18T07:13:13.329 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:19.3.0-4953.g82e3d596.el9.x86_64 3/131 2024-09-18T07:13:13.329 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:19.3.0-4953.g82e3d596.el9.x86_64 4/131 2024-09-18T07:13:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4953.g82e3d 5/131 2024-09-18T07:13:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:19.3.0-4953.g82e3d596.el9.x86_64 6/131 2024-09-18T07:13:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:19.3.0-4953.g82e3d596.el9.x86_64 7/131 2024-09-18T07:13:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:19.3.0-4953.g82e3d596.el9.x86_64 8/131 2024-09-18T07:13:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:19.3.0-4953.g82e3d596.el9.x86_64 9/131 2024-09-18T07:13:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:19.3.0-4953.g82e3d596.el9.x86_64 10/131 2024-09-18T07:13:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:19.3.0-4953.g82e3d596.el9.x86_64 11/131 2024-09-18T07:13:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:19.3.0-4953.g82e3d596.el9.x86_64 12/131 2024-09-18T07:13:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:19.3.0-4953.g82e3d596.el9.x86_ 13/131 2024-09-18T07:13:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:19.3.0-4953.g82e3d596.el9.x86_64 14/131 2024-09-18T07:13:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:19.3.0-4953.g82e3d596.el9.x86_64 15/131 2024-09-18T07:13:13.330 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:19.3.0-4953.g82e3d596.el9.x86_6 16/131 2024-09-18T07:13:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:19.3.0-4953.g82e3d596.el9.x86 17/131 2024-09-18T07:13:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:19.3.0-4953.g82e3d596.el9.x86_64 18/131 2024-09-18T07:13:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4953.g82e3d596.el 19/131 2024-09-18T07:13:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:19.3.0-4953.g82e3d596.el9. 20/131 2024-09-18T07:13:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:19.3.0-4953.g82e3d596.el9.x86_6 21/131 2024-09-18T07:13:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:19.3.0-4953.g82e3d596.el9.x86_64 22/131 2024-09-18T07:13:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:19.3.0-4953.g82e3d596.el9.x86_64 23/131 2024-09-18T07:13:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:19.3.0-4953.g82e3d596.el9.x86_64 24/131 2024-09-18T07:13:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:19.3.0-4953.g82e3d596.el9.x86_64 25/131 2024-09-18T07:13:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:19.3.0-4953.g82e3d596.el9.x86_64 26/131 2024-09-18T07:13:13.331 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:19.3.0-4953.g82e3d596.el9.x86_64 27/131 2024-09-18T07:13:13.332 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4953.g82e3d596. 28/131 2024-09-18T07:13:13.332 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4953.g82e3d596.el9.noa 29/131 2024-09-18T07:13:13.333 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4953.g82e3d596.el9.n 30/131 2024-09-18T07:13:13.341 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4953.g82e 31/131 2024-09-18T07:13:13.341 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4953.g82e3d596.el 32/131 2024-09-18T07:13:13.341 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4953.g82e3d596.el9.noarch 33/131 2024-09-18T07:13:13.341 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4953.g82e3d596.e 34/131 2024-09-18T07:13:13.341 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-volume-2:19.3.0-4953.g82e3d596.el9.noarch 35/131 2024-09-18T07:13:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:19.3.0-4953.g82e3d596.el9.noarch 36/131 2024-09-18T07:13:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 37/131 2024-09-18T07:13:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 38/131 2024-09-18T07:13:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 39/131 2024-09-18T07:13:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/131 2024-09-18T07:13:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/131 2024-09-18T07:13:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/131 2024-09-18T07:13:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/131 2024-09-18T07:13:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/131 2024-09-18T07:13:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 45/131 2024-09-18T07:13:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 46/131 2024-09-18T07:13:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 47/131 2024-09-18T07:13:13.342 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 48/131 2024-09-18T07:13:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 49/131 2024-09-18T07:13:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 50/131 2024-09-18T07:13:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 51/131 2024-09-18T07:13:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 52/131 2024-09-18T07:13:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 53/131 2024-09-18T07:13:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 54/131 2024-09-18T07:13:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-5.4.4-4.el9.x86_64 55/131 2024-09-18T07:13:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 56/131 2024-09-18T07:13:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 57/131 2024-09-18T07:13:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/131 2024-09-18T07:13:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 59/131 2024-09-18T07:13:13.343 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 60/131 2024-09-18T07:13:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 61/131 2024-09-18T07:13:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 62/131 2024-09-18T07:13:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 63/131 2024-09-18T07:13:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 64/131 2024-09-18T07:13:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 65/131 2024-09-18T07:13:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 66/131 2024-09-18T07:13:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 67/131 2024-09-18T07:13:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 68/131 2024-09-18T07:13:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 69/131 2024-09-18T07:13:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 70/131 2024-09-18T07:13:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 71/131 2024-09-18T07:13:13.344 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 72/131 2024-09-18T07:13:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 73/131 2024-09-18T07:13:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 74/131 2024-09-18T07:13:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 75/131 2024-09-18T07:13:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 76/131 2024-09-18T07:13:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 77/131 2024-09-18T07:13:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 78/131 2024-09-18T07:13:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 79/131 2024-09-18T07:13:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 80/131 2024-09-18T07:13:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 81/131 2024-09-18T07:13:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 82/131 2024-09-18T07:13:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : sqlite-devel-3.34.1-7.el9.x86_64 83/131 2024-09-18T07:13:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/131 2024-09-18T07:13:13.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 85/131 2024-09-18T07:13:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 86/131 2024-09-18T07:13:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 87/131 2024-09-18T07:13:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 88/131 2024-09-18T07:13:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 89/131 2024-09-18T07:13:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 90/131 2024-09-18T07:13:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 91/131 2024-09-18T07:13:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 92/131 2024-09-18T07:13:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 93/131 2024-09-18T07:13:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 94/131 2024-09-18T07:13:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 95/131 2024-09-18T07:13:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 96/131 2024-09-18T07:13:13.346 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 97/131 2024-09-18T07:13:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 98/131 2024-09-18T07:13:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 99/131 2024-09-18T07:13:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 100/131 2024-09-18T07:13:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 101/131 2024-09-18T07:13:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 102/131 2024-09-18T07:13:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 103/131 2024-09-18T07:13:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 104/131 2024-09-18T07:13:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 105/131 2024-09-18T07:13:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 106/131 2024-09-18T07:13:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 107/131 2024-09-18T07:13:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 108/131 2024-09-18T07:13:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 109/131 2024-09-18T07:13:13.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 110/131 2024-09-18T07:13:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 111/131 2024-09-18T07:13:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 112/131 2024-09-18T07:13:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 113/131 2024-09-18T07:13:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 114/131 2024-09-18T07:13:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 115/131 2024-09-18T07:13:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 116/131 2024-09-18T07:13:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 117/131 2024-09-18T07:13:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 118/131 2024-09-18T07:13:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 119/131 2024-09-18T07:13:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 120/131 2024-09-18T07:13:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 121/131 2024-09-18T07:13:13.348 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 122/131 2024-09-18T07:13:13.349 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 123/131 2024-09-18T07:13:13.349 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 124/131 2024-09-18T07:13:13.349 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 125/131 2024-09-18T07:13:13.349 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : re2-1:20211101-20.el9.x86_64 126/131 2024-09-18T07:13:13.349 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 127/131 2024-09-18T07:13:13.349 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:19.3.0-4953.g82e3d596.el9.x86_64 128/131 2024-09-18T07:13:13.349 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 129/131 2024-09-18T07:13:13.349 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:19.3.0-4953.g82e3d596.el9.x86_64 130/131 2024-09-18T07:13:13.482 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 131/131 2024-09-18T07:13:13.482 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:13.482 INFO:teuthology.orchestra.run.smithi172.stdout:Upgraded: 2024-09-18T07:13:13.482 INFO:teuthology.orchestra.run.smithi172.stdout: librados2-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.483 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.483 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-18T07:13:13.483 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-18T07:13:13.483 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-18T07:13:13.483 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.483 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.483 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.483 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.483 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:13.483 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.483 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.483 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.484 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:13.484 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:13.484 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:13.484 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:13.484 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:13.484 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.484 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.484 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:13.484 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.484 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.484 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.484 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:13.484 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:13.485 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-18T07:13:13.485 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-18T07:13:13.485 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-18T07:13:13.485 INFO:teuthology.orchestra.run.smithi172.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-18T07:13:13.485 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-18T07:13:13.485 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-18T07:13:13.485 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-18T07:13:13.485 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-18T07:13:13.485 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.485 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.485 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.486 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-18T07:13:13.486 INFO:teuthology.orchestra.run.smithi172.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-18T07:13:13.486 INFO:teuthology.orchestra.run.smithi172.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-18T07:13:13.486 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-18T07:13:13.486 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-18T07:13:13.486 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-18T07:13:13.486 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.486 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.486 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-18T07:13:13.486 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.486 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-18T07:13:13.486 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-18T07:13:13.487 INFO:teuthology.orchestra.run.smithi172.stdout: lua-5.4.4-4.el9.x86_64 2024-09-18T07:13:13.487 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-18T07:13:13.487 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-18T07:13:13.487 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-18T07:13:13.487 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-18T07:13:13.487 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-18T07:13:13.487 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-18T07:13:13.487 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-18T07:13:13.487 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-18T07:13:13.487 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-18T07:13:13.487 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-18T07:13:13.487 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-18T07:13:13.487 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.488 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.488 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.488 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-18T07:13:13.488 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-18T07:13:13.488 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-18T07:13:13.488 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-18T07:13:13.488 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-18T07:13:13.488 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-18T07:13:13.488 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-18T07:13:13.488 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-18T07:13:13.488 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-18T07:13:13.488 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-18T07:13:13.489 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-18T07:13:13.489 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-18T07:13:13.489 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-18T07:13:13.489 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-18T07:13:13.489 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-18T07:13:13.489 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-18T07:13:13.489 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-18T07:13:13.489 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-18T07:13:13.489 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-18T07:13:13.489 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-18T07:13:13.489 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-18T07:13:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-18T07:13:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-18T07:13:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-18T07:13:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-18T07:13:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-18T07:13:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-18T07:13:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-18T07:13:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply-3.11-14.el9.noarch 2024-09-18T07:13:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-18T07:13:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-18T07:13:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-18T07:13:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-18T07:13:13.490 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-18T07:13:13.491 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-18T07:13:13.491 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-18T07:13:13.491 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-18T07:13:13.491 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.491 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.491 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-18T07:13:13.491 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-18T07:13:13.491 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-18T07:13:13.491 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.491 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-18T07:13:13.491 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-18T07:13:13.491 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-18T07:13:13.491 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-18T07:13:13.492 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-18T07:13:13.492 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-18T07:13:13.492 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-18T07:13:13.492 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-18T07:13:13.492 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-18T07:13:13.492 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-18T07:13:13.492 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-18T07:13:13.492 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-18T07:13:13.492 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-18T07:13:13.492 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-18T07:13:13.492 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.492 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.493 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:13.493 INFO:teuthology.orchestra.run.smithi172.stdout: re2-1:20211101-20.el9.x86_64 2024-09-18T07:13:13.493 INFO:teuthology.orchestra.run.smithi172.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-18T07:13:13.493 INFO:teuthology.orchestra.run.smithi172.stdout: sqlite-3.34.1-7.el9.x86_64 2024-09-18T07:13:13.493 INFO:teuthology.orchestra.run.smithi172.stdout: sqlite-devel-3.34.1-7.el9.x86_64 2024-09-18T07:13:13.493 INFO:teuthology.orchestra.run.smithi172.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-18T07:13:13.493 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-18T07:13:13.493 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:13.494 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-18T07:13:13.806 DEBUG:teuthology.parallel:result is None 2024-09-18T07:13:15.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 131/131 2024-09-18T07:13:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:15.586 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2024-09-18T07:13:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.586 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-18T07:13:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-18T07:13:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-18T07:13:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:15.586 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.587 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:19.3.0-4953.g82e3d596.el9.noarch 2024-09-18T07:13:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-18T07:13:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-18T07:13:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-18T07:13:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-18T07:13:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-18T07:13:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-18T07:13:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-18T07:13:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-18T07:13:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.588 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-18T07:13:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-18T07:13:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-18T07:13:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-18T07:13:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-18T07:13:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-18T07:13:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-18T07:13:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-18T07:13:15.589 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-18T07:13:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: lua-5.4.4-4.el9.x86_64 2024-09-18T07:13:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-18T07:13:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-18T07:13:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-18T07:13:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-18T07:13:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-18T07:13:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-18T07:13:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-18T07:13:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-18T07:13:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-18T07:13:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-18T07:13:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-18T07:13:15.590 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-18T07:13:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-18T07:13:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-18T07:13:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-18T07:13:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-18T07:13:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-18T07:13:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-18T07:13:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-18T07:13:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-18T07:13:15.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-18T07:13:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-18T07:13:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-18T07:13:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-18T07:13:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-18T07:13:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-18T07:13:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-18T07:13:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-18T07:13:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-18T07:13:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-18T07:13:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-18T07:13:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-18T07:13:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-18T07:13:15.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-18T07:13:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-18T07:13:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-18T07:13:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-18T07:13:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-18T07:13:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-18T07:13:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply-3.11-14.el9.noarch 2024-09-18T07:13:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-18T07:13:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-18T07:13:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-18T07:13:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-18T07:13:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-18T07:13:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-18T07:13:15.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-18T07:13:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-18T07:13:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-18T07:13:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-18T07:13:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-18T07:13:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-18T07:13:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-18T07:13:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-18T07:13:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-18T07:13:15.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-18T07:13:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-18T07:13:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-18T07:13:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-18T07:13:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-18T07:13:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-18T07:13:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-18T07:13:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-18T07:13:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-18T07:13:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-18T07:13:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.595 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:19.3.0-4953.g82e3d596.el9.x86_64 2024-09-18T07:13:15.596 INFO:teuthology.orchestra.run.smithi070.stdout: re2-1:20211101-20.el9.x86_64 2024-09-18T07:13:15.596 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-18T07:13:15.596 INFO:teuthology.orchestra.run.smithi070.stdout: sqlite-3.34.1-7.el9.x86_64 2024-09-18T07:13:15.596 INFO:teuthology.orchestra.run.smithi070.stdout: sqlite-devel-3.34.1-7.el9.x86_64 2024-09-18T07:13:15.596 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-18T07:13:15.596 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-18T07:13:15.596 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:15.596 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-18T07:13:15.932 DEBUG:teuthology.parallel:result is None 2024-09-18T07:13:15.933 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=82e3d5962b49b7d08ddbe09b754f372dd36156d5 2024-09-18T07:13:16.088 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-18T07:13:16.109 INFO:teuthology.orchestra.run.smithi070.stdout:19.3.0-4953.g82e3d596.el9 2024-09-18T07:13:16.110 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4953.g82e3d596.el9 2024-09-18T07:13:16.110 INFO:teuthology.task.install:The correct ceph version 19.3.0-4953.g82e3d596 is installed. 2024-09-18T07:13:16.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=82e3d5962b49b7d08ddbe09b754f372dd36156d5 2024-09-18T07:13:16.262 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-18T07:13:16.288 INFO:teuthology.orchestra.run.smithi172.stdout:19.3.0-4953.g82e3d596.el9 2024-09-18T07:13:16.289 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4953.g82e3d596.el9 2024-09-18T07:13:16.289 INFO:teuthology.task.install:The correct ceph version 19.3.0-4953.g82e3d596 is installed. 2024-09-18T07:13:16.290 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-18T07:13:16.291 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:16.291 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T07:13:16.317 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:16.317 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T07:13:16.360 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-18T07:13:16.360 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:16.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T07:13:16.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T07:13:16.451 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:16.451 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T07:13:16.479 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T07:13:16.545 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-18T07:13:16.546 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:16.546 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T07:13:16.570 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T07:13:16.635 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:16.635 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T07:13:16.661 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T07:13:16.726 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-18T07:13:16.727 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:16.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T07:13:16.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T07:13:16.815 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:16.816 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T07:13:16.841 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T07:13:16.905 INFO:teuthology.run_tasks:Running task ceph... 2024-09-18T07:13:17.237 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-18T07:13:17.237 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /var/log/ceph 2024-09-18T07:13:17.240 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /var/log/ceph 2024-09-18T07:13:17.264 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-18T07:13:17.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T07:13:17.306 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T07:13:17.330 INFO:tasks.ceph:Creating extra log directories... 2024-09-18T07:13:17.330 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T07:13:17.372 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T07:13:17.408 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-18T07:13:17.409 INFO:tasks.ceph:config {'conf': {'client': {'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2024-09-18T07:13:17.409 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi/7910278', 'branch': 'main', 'description': 'rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} tasks/libcephsqlite}', 'email': None, 'first_in_suite': False, 'job_id': '7910278', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled'], 'mon_bind_msgr2': False, 'sha1': '82e3d5962b49b7d08ddbe09b754f372dd36156d5'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '82e3d5962b49b7d08ddbe09b754f372dd36156d5'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779'}}, 'owner': 'scheduled_nmordech@teuthology', 'priority': 50, 'repo': 'https://github.com/ceph/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 4523, 'sha1': '82e3d5962b49b7d08ddbe09b754f372dd36156d5', 'sleep_before_teardown': 0, 'suite': 'rados:basic', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779', 'targets': {'smithi070.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQAz9ntCMJqZQnLilFX1KjpYnTB2hDn3MLfk3XGbitpB1/BNoar2fSwEkkaw4D//uMkMEtKtBWGupn3wyPJB2w=', 'smithi172.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGqDrmQeae9RbqQO8dH0iE6uRpgU54gpeMteFMngd6EDPAt3MURb9IohvkN0e2s8ELnsb258r8liooylziVa++Q='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': '82e3d5962b49b7d08ddbe09b754f372dd36156d5'}}, {'ceph': None}, {'exec': {'client.0': ['ceph osd pool create cephsqlite', "ceph auth get-or-create client.libcephsqlite mon 'profile simple-rados-client-with-blocklist' osd 'allow rwx pool=cephsqlite' >> /etc/ceph/ceph.keyring"]}}, {'exec': {'client.0': ['ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr']}}, {'workunit': {'clients': {'client.0': ['rados/test_libcephsqlite.sh cephsqlite']}, 'env': {'CEPH_ARGS': '--id libcephsqlite --no-log-to-stderr'}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-18_06:58:32', 'tube': 'smithi', 'user': 'nmordech', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-18T07:13:17.409 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T07:13:17.437 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T07:13:17.452 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T07:13:17.493 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T07:13:17.517 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:17.517 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T07:13:17.571 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T07:13:17.571 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2024-09-18T07:13:17.626 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T07:13:17.626 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T07:13:17.627 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-18T07:13:17.627 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T07:13:17.627 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T07:13:17.627 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-18 07:13:07.777782291 +0000 2024-09-18T07:13:17.627 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-18 07:11:13.495184552 +0000 2024-09-18T07:13:17.627 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-18 07:11:13.495184552 +0000 2024-09-18T07:13:17.627 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-18 07:11:13.495184552 +0000 2024-09-18T07:13:17.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T07:13:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-18T07:13:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-18T07:13:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 8.183e-05 s, 6.3 MB/s 2024-09-18T07:13:17.690 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T07:13:17.746 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2024-09-18T07:13:17.802 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T07:13:17.802 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T07:13:17.802 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-18T07:13:17.802 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T07:13:17.802 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T07:13:17.802 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-18 07:13:07.777782291 +0000 2024-09-18T07:13:17.802 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-18 07:11:13.827174715 +0000 2024-09-18T07:13:17.802 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-18 07:11:13.827174715 +0000 2024-09-18T07:13:17.803 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-18 07:11:13.827174715 +0000 2024-09-18T07:13:17.803 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T07:13:17.865 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-18T07:13:17.865 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-18T07:13:17.866 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000122594 s, 4.2 MB/s 2024-09-18T07:13:17.867 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T07:13:17.923 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2024-09-18T07:13:17.979 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T07:13:17.979 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T07:13:17.979 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-18T07:13:17.979 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T07:13:17.979 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T07:13:17.979 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-18 07:13:07.778782261 +0000 2024-09-18T07:13:17.979 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-18 07:11:14.143165352 +0000 2024-09-18T07:13:17.980 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-18 07:11:14.143165352 +0000 2024-09-18T07:13:17.980 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-18 07:11:14.143165352 +0000 2024-09-18T07:13:17.980 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T07:13:18.042 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-18T07:13:18.042 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-18T07:13:18.042 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 8.0352e-05 s, 6.4 MB/s 2024-09-18T07:13:18.043 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T07:13:18.099 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2024-09-18T07:13:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T07:13:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T07:13:18.155 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-18T07:13:18.155 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T07:13:18.155 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T07:13:18.155 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-18 07:13:07.778782261 +0000 2024-09-18T07:13:18.155 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-18 07:11:14.463155871 +0000 2024-09-18T07:13:18.155 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-18 07:11:14.463155871 +0000 2024-09-18T07:13:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-18 07:11:14.463155871 +0000 2024-09-18T07:13:18.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T07:13:18.218 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-18T07:13:18.218 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-18T07:13:18.218 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 7.7005e-05 s, 6.6 MB/s 2024-09-18T07:13:18.219 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T07:13:18.275 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T07:13:18.276 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:18.276 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T07:13:18.294 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T07:13:18.294 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2024-09-18T07:13:18.351 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T07:13:18.351 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T07:13:18.351 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-18T07:13:18.351 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T07:13:18.351 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T07:13:18.351 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-18 07:13:05.649286666 +0000 2024-09-18T07:13:18.351 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-18 07:11:17.482014460 +0000 2024-09-18T07:13:18.351 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-18 07:11:17.482014460 +0000 2024-09-18T07:13:18.352 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-18 07:11:17.482014460 +0000 2024-09-18T07:13:18.352 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T07:13:18.416 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-18T07:13:18.416 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-18T07:13:18.416 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000246233 s, 2.1 MB/s 2024-09-18T07:13:18.417 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T07:13:18.475 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2024-09-18T07:13:18.531 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T07:13:18.531 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T07:13:18.531 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-18T07:13:18.531 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T07:13:18.531 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T07:13:18.531 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-18 07:13:05.649286666 +0000 2024-09-18T07:13:18.531 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-18 07:11:17.823006060 +0000 2024-09-18T07:13:18.531 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-18 07:11:17.823006060 +0000 2024-09-18T07:13:18.531 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-18 07:11:17.823006060 +0000 2024-09-18T07:13:18.532 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T07:13:18.595 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-18T07:13:18.595 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-18T07:13:18.595 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000252917 s, 2.0 MB/s 2024-09-18T07:13:18.597 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T07:13:18.656 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2024-09-18T07:13:18.712 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T07:13:18.712 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T07:13:18.712 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-18T07:13:18.712 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T07:13:18.712 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T07:13:18.712 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-18 07:13:05.649286666 +0000 2024-09-18T07:13:18.712 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-18 07:11:18.155997858 +0000 2024-09-18T07:13:18.712 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-18 07:11:18.155997858 +0000 2024-09-18T07:13:18.712 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-18 07:11:18.155997858 +0000 2024-09-18T07:13:18.713 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T07:13:18.776 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-18T07:13:18.776 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-18T07:13:18.776 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000218963 s, 2.3 MB/s 2024-09-18T07:13:18.777 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T07:13:18.834 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2024-09-18T07:13:18.889 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T07:13:18.889 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T07:13:18.889 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-18T07:13:18.889 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T07:13:18.890 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T07:13:18.890 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-18 07:13:05.650286642 +0000 2024-09-18T07:13:18.890 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-18 07:11:18.473990026 +0000 2024-09-18T07:13:18.890 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-18 07:11:18.473990026 +0000 2024-09-18T07:13:18.890 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-18 07:11:18.473990026 +0000 2024-09-18T07:13:18.890 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T07:13:18.953 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-18T07:13:18.953 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-18T07:13:18.953 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000218113 s, 2.3 MB/s 2024-09-18T07:13:18.954 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T07:13:19.011 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T07:13:19.011 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-18T07:13:19.011 INFO:tasks.ceph:Generating config... 2024-09-18T07:13:19.027 INFO:tasks.ceph:[client] debug cephsqlite = 20 2024-09-18T07:13:19.027 INFO:tasks.ceph:[client] debug client = 20 2024-09-18T07:13:19.027 INFO:tasks.ceph:[client] debug ms = 1 2024-09-18T07:13:19.027 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-18T07:13:19.027 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-18T07:13:19.027 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-09-18T07:13:19.027 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-18T07:13:19.027 INFO:tasks.ceph:[global] ms type = async 2024-09-18T07:13:19.027 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-18T07:13:19.027 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-18T07:13:19.027 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-18T07:13:19.027 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-18T07:13:19.028 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-18T07:13:19.028 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-09-18T07:13:19.028 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-18T07:13:19.028 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-18T07:13:19.028 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-18T07:13:19.028 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-18T07:13:19.028 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-09-18T07:13:19.028 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-18T07:13:19.028 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-18T07:13:19.028 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-18T07:13:19.028 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-18T07:13:19.028 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-18T07:13:19.029 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-18T07:13:19.030 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-18T07:13:19.030 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-18T07:13:19.030 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-18T07:13:19.030 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-18T07:13:19.030 INFO:tasks.ceph:Setting up mon.a... 2024-09-18T07:13:19.030 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-18T07:13:19.082 INFO:teuthology.orchestra.run.smithi070.stdout:creating /etc/ceph/ceph.keyring 2024-09-18T07:13:19.085 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-18T07:13:19.128 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T07:13:19.193 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.70:6789]'), ('mon.c', '[v1:172.21.15.70:6790]'), ('mon.b', '[v1:172.21.15.172:6789]')] 2024-09-18T07:13:19.194 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.70:6789],[v1:172.21.15.70:6790],[v1:172.21.15.172:6789]', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-18T07:13:19.195 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:19.195 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T07:13:19.248 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.70:6789]' --addv c '[v1:172.21.15.70:6790]' --addv b '[v1:172.21.15.172:6789]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:19.335 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:19.336 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: generated fsid 1fa528ad-1630-4e95-9001-978a53f2a42c 2024-09-18T07:13:19.336 INFO:teuthology.orchestra.run.smithi070.stdout:setting min_mon_release = quincy 2024-09-18T07:13:19.336 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 0 2024-09-18T07:13:19.336 INFO:teuthology.orchestra.run.smithi070.stdout:fsid 1fa528ad-1630-4e95-9001-978a53f2a42c 2024-09-18T07:13:19.336 INFO:teuthology.orchestra.run.smithi070.stdout:last_changed 2024-09-18T07:13:19.334097+0000 2024-09-18T07:13:19.336 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-09-18T07:13:19.334097+0000 2024-09-18T07:13:19.336 INFO:teuthology.orchestra.run.smithi070.stdout:min_mon_release 17 (quincy) 2024-09-18T07:13:19.336 INFO:teuthology.orchestra.run.smithi070.stdout:election_strategy: 1 2024-09-18T07:13:19.336 INFO:teuthology.orchestra.run.smithi070.stdout:0: v1:172.21.15.70:6789/0 mon.a 2024-09-18T07:13:19.336 INFO:teuthology.orchestra.run.smithi070.stdout:1: v1:172.21.15.172:6789/0 mon.b 2024-09-18T07:13:19.337 INFO:teuthology.orchestra.run.smithi070.stdout:2: v1:172.21.15.70:6790/0 mon.c 2024-09-18T07:13:19.337 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-18T07:13:19.338 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T07:13:19.353 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1fa528ad-1630-4e95-9001-978a53f2a42c... 2024-09-18T07:13:19.354 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T07:13:19.395 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T07:13:19.434 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: chdir = "" 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: auth supported = cephx 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: filestore xattr use omap = true 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: mon clock drift allowed = 1.000 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: osd crush chooseleaf type = 0 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: auth debug = true 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on old message = true 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on bug = true 2024-09-18T07:13:19.435 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.436 INFO:teuthology.orchestra.run.smithi070.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T07:13:19.436 INFO:teuthology.orchestra.run.smithi070.stdout: mon pg warn max object skew = 0 2024-09-18T07:13:19.436 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.436 INFO:teuthology.orchestra.run.smithi070.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T07:13:19.436 INFO:teuthology.orchestra.run.smithi070.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T07:13:19.436 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.436 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default size = 2 2024-09-18T07:13:19.436 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.436 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow primary affinity = true 2024-09-18T07:13:19.436 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow pg remap = true 2024-09-18T07:13:19.436 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on legacy crush tunables = false 2024-09-18T07:13:19.436 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on crush straw calc version zero = false 2024-09-18T07:13:19.436 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on no sortbitwise = false 2024-09-18T07:13:19.437 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on osd down out interval zero = false 2024-09-18T07:13:19.437 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on too few osds = false 2024-09-18T07:13:19.437 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T07:13:19.437 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T07:13:19.437 INFO:teuthology.orchestra.run.smithi070.stdout: mon_allow_pool_size_one = true 2024-09-18T07:13:19.437 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.437 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T07:13:19.437 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.437 INFO:teuthology.orchestra.run.smithi070.stdout: osd default data pool replay window = 5 2024-09-18T07:13:19.437 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.437 INFO:teuthology.orchestra.run.smithi070.stdout: mon allow pool delete = true 2024-09-18T07:13:19.437 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: mon cluster log file level = debug 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: debug asserts on shutdown = true 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: mon health detail to clog = false 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: mon host = "[v1:172.21.15.70:6789],[v1:172.21.15.70:6790],[v1:172.21.15.172:6789]" 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: mon client directed command retry = 5 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: mon election default strategy = 1 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: ms bind msgr2 = False 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: ms inject socket failures = 5000 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: ms type = async 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = 1fa528ad-1630-4e95-9001-978a53f2a42c 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout:[osd] 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: osd journal size = 100 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.438 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub load threshold = 5.0 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub max interval = 600 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: osd mclock profile = high_recovery_ops 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: osd recover clone overlap = true 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: osd recovery max chunk = 1048576 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug shutdown = true 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug op order = true 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug verify stray on activate = true 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug trim objects = true 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: osd open classes on start = true 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug pg log writeout = true 2024-09-18T07:13:19.439 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.440 INFO:teuthology.orchestra.run.smithi070.stdout: osd deep scrub update digest min age = 30 2024-09-18T07:13:19.440 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.440 INFO:teuthology.orchestra.run.smithi070.stdout: osd map max advance = 10 2024-09-18T07:13:19.440 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.440 INFO:teuthology.orchestra.run.smithi070.stdout: journal zero on create = true 2024-09-18T07:13:19.440 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.440 INFO:teuthology.orchestra.run.smithi070.stdout: filestore ondisk finisher threads = 3 2024-09-18T07:13:19.440 INFO:teuthology.orchestra.run.smithi070.stdout: filestore apply finisher threads = 3 2024-09-18T07:13:19.440 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.440 INFO:teuthology.orchestra.run.smithi070.stdout: bdev debug aio = true 2024-09-18T07:13:19.440 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug misdirected ops = true 2024-09-18T07:13:19.440 INFO:teuthology.orchestra.run.smithi070.stdout: bdev async discard = True 2024-09-18T07:13:19.440 INFO:teuthology.orchestra.run.smithi070.stdout: bdev enable discard = True 2024-09-18T07:13:19.441 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore allocator = bitmap 2024-09-18T07:13:19.441 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore block size = 96636764160 2024-09-18T07:13:19.441 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore fsck on mount = True 2024-09-18T07:13:19.441 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore zero block detection = True 2024-09-18T07:13:19.441 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluefs = 20 2024-09-18T07:13:19.441 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluestore = 20 2024-09-18T07:13:19.441 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-18T07:13:19.441 INFO:teuthology.orchestra.run.smithi070.stdout: debug osd = 20 2024-09-18T07:13:19.441 INFO:teuthology.orchestra.run.smithi070.stdout: debug rocksdb = 10 2024-09-18T07:13:19.441 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T07:13:19.441 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd full ratio = 0.9 2024-09-18T07:13:19.441 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd nearfull ratio = 0.8 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug verify cached snaps = True 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug verify missing on start = True 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: osd failsafe full ratio = 0.95 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: osd mclock override recovery settings = True 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: osd objectstore = bluestore 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: osd op queue = debug_random 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: osd op queue cut off = debug_random 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: osd shutdown pgref assert = True 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr] 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: debug mgr = 20 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2024-09-18T07:13:19.442 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2024-09-18T07:13:19.443 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2024-09-18T07:13:19.443 INFO:teuthology.orchestra.run.smithi070.stdout: mgr/telemetry/nag = false 2024-09-18T07:13:19.443 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.443 INFO:teuthology.orchestra.run.smithi070.stdout:[mon] 2024-09-18T07:13:19.443 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-18T07:13:19.443 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2024-09-18T07:13:19.443 INFO:teuthology.orchestra.run.smithi070.stdout: debug paxos = 20 2024-09-18T07:13:19.443 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2024-09-18T07:13:19.443 INFO:teuthology.orchestra.run.smithi070.stdout: mon data avail warn = 5 2024-09-18T07:13:19.443 INFO:teuthology.orchestra.run.smithi070.stdout: mon mgr mkfs grace = 240 2024-09-18T07:13:19.443 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2024-09-18T07:13:19.443 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd reporter subtree level = osd 2024-09-18T07:13:19.443 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd prime pg temp = true 2024-09-18T07:13:19.444 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2024-09-18T07:13:19.444 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.444 INFO:teuthology.orchestra.run.smithi070.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T07:13:19.444 INFO:teuthology.orchestra.run.smithi070.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T07:13:19.444 INFO:teuthology.orchestra.run.smithi070.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T07:13:19.444 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.444 INFO:teuthology.orchestra.run.smithi070.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T07:13:19.444 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T07:13:19.444 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T07:13:19.444 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.444 INFO:teuthology.orchestra.run.smithi070.stdout: # 1m isn't quite enough 2024-09-18T07:13:19.444 INFO:teuthology.orchestra.run.smithi070.stdout: mon_down_mkfs_grace = 2m 2024-09-18T07:13:19.444 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_filestore_osds = false 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: mon min osdmap epochs = 50 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: mon osdmap full prune interval = 2 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: mon osdmap full prune min = 15 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: mon osdmap full prune txsize = 2 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: mon scrub interval = 300 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: paxos service trim min = 10 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout:[client] 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: rgw cache enabled = true 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable ops log = true 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable usage log = true 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T07:13:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T07:13:19.446 INFO:teuthology.orchestra.run.smithi070.stdout: debug cephsqlite = 20 2024-09-18T07:13:19.446 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2024-09-18T07:13:19.446 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-18T07:13:19.446 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.a] 2024-09-18T07:13:19.446 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.c] 2024-09-18T07:13:19.446 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.b] 2024-09-18T07:13:19.446 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2024-09-18T07:13:19.446 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2024-09-18T07:13:19.447 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T07:13:19.447 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2024-09-18T07:13:19.447 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.447 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2024-09-18T07:13:19.447 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.447 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2024-09-18T07:13:19.447 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.447 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2024-09-18T07:13:19.447 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2024-09-18T07:13:19.447 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.447 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2024-09-18T07:13:19.447 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2024-09-18T07:13:19.448 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2024-09-18T07:13:19.449 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2024-09-18T07:13:19.449 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2024-09-18T07:13:19.449 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T07:13:19.449 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T07:13:19.449 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2024-09-18T07:13:19.449 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.449 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T07:13:19.449 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.449 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2024-09-18T07:13:19.449 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.449 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2024-09-18T07:13:19.449 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.449 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "[v1:172.21.15.70:6789],[v1:172.21.15.70:6790],[v1:172.21.15.172:6789]" 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout: mon client directed command retry = 5 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout: mon election default strategy = 1 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout: ms bind msgr2 = False 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout: ms inject socket failures = 5000 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout: ms type = async 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = 1fa528ad-1630-4e95-9001-978a53f2a42c 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.450 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug trim objects = true 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.451 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: bdev async discard = True 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: bdev enable discard = True 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore allocator = bitmap 2024-09-18T07:13:19.452 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2024-09-18T07:13:19.453 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2024-09-18T07:13:19.453 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore zero block detection = True 2024-09-18T07:13:19.453 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 20 2024-09-18T07:13:19.453 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 20 2024-09-18T07:13:19.453 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-09-18T07:13:19.456 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2024-09-18T07:13:19.456 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 10 2024-09-18T07:13:19.456 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T07:13:19.456 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2024-09-18T07:13:19.456 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2024-09-18T07:13:19.456 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify cached snaps = True 2024-09-18T07:13:19.457 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify missing on start = True 2024-09-18T07:13:19.457 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2024-09-18T07:13:19.457 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock override recovery settings = True 2024-09-18T07:13:19.457 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2024-09-18T07:13:19.457 INFO:teuthology.orchestra.run.smithi172.stdout: osd op queue = debug_random 2024-09-18T07:13:19.457 INFO:teuthology.orchestra.run.smithi172.stdout: osd op queue cut off = debug_random 2024-09-18T07:13:19.457 INFO:teuthology.orchestra.run.smithi172.stdout: osd shutdown pgref assert = True 2024-09-18T07:13:19.457 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.457 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2024-09-18T07:13:19.457 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-09-18T07:13:19.457 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2024-09-18T07:13:19.457 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-09-18T07:13:19.457 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-09-18T07:13:19.458 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-09-18T07:13:19.458 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-09-18T07:13:19.458 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2024-09-18T07:13:19.458 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.458 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2024-09-18T07:13:19.458 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-09-18T07:13:19.458 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-09-18T07:13:19.458 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2024-09-18T07:13:19.458 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-09-18T07:13:19.458 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2024-09-18T07:13:19.458 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2024-09-18T07:13:19.458 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2024-09-18T07:13:19.459 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2024-09-18T07:13:19.460 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.460 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2024-09-18T07:13:19.460 INFO:teuthology.orchestra.run.smithi172.stdout: mon min osdmap epochs = 50 2024-09-18T07:13:19.460 INFO:teuthology.orchestra.run.smithi172.stdout: mon osdmap full prune interval = 2 2024-09-18T07:13:19.460 INFO:teuthology.orchestra.run.smithi172.stdout: mon osdmap full prune min = 15 2024-09-18T07:13:19.460 INFO:teuthology.orchestra.run.smithi172.stdout: mon osdmap full prune txsize = 2 2024-09-18T07:13:19.460 INFO:teuthology.orchestra.run.smithi172.stdout: mon scrub interval = 300 2024-09-18T07:13:19.460 INFO:teuthology.orchestra.run.smithi172.stdout: paxos service trim min = 10 2024-09-18T07:13:19.460 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-18T07:13:19.460 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2024-09-18T07:13:19.460 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2024-09-18T07:13:19.460 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2024-09-18T07:13:19.460 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2024-09-18T07:13:19.461 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T07:13:19.461 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T07:13:19.461 INFO:teuthology.orchestra.run.smithi172.stdout: debug cephsqlite = 20 2024-09-18T07:13:19.461 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-09-18T07:13:19.461 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-09-18T07:13:19.461 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2024-09-18T07:13:19.461 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2024-09-18T07:13:19.461 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2024-09-18T07:13:19.462 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-18T07:13:19.463 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-18T07:13:19.557 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-18T07:13:19.557 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:19.557 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-18T07:13:19.572 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:19.572 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-18T07:13:19.627 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi070.front.sepia.ceph.com 2024-09-18T07:13:19.627 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:19.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T07:13:19.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T07:13:19.701 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:19.701 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:19.757 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi172.front.sepia.ceph.com 2024-09-18T07:13:19.757 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:19.757 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T07:13:19.757 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T07:13:19.793 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:19.793 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:19.849 INFO:tasks.ceph:Setting up mon nodes... 2024-09-18T07:13:19.849 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-18T07:13:19.849 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T07:13:19.899 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T07:13:19.902 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T07:13:19.979 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T07:13:19.981 INFO:tasks.ceph:Setting up mds nodes... 2024-09-18T07:13:19.982 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-18T07:13:19.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-18T07:13:20.023 INFO:teuthology.orchestra.run.smithi070.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-18T07:13:20.035 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-18T07:13:20.076 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-18T07:13:20.090 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-18T07:13:20.090 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-18T07:13:20.090 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-18T07:13:20.115 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T07:13:20.115 INFO:tasks.ceph:role: osd.0 2024-09-18T07:13:20.115 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-18T07:13:20.115 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T07:13:20.179 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T07:13:20.179 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T07:13:20.179 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T07:13:20.180 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T07:13:20.180 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T07:13:20.180 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-18T07:13:20.180 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T07:13:20.180 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T07:13:20.180 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T07:13:20.180 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T07:13:20.212 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-18T07:13:20.214 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-18T07:13:20.214 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-18T07:13:20.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-18T07:13:20.350 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-18T07:13:20.415 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T07:13:20.415 INFO:tasks.ceph:role: osd.1 2024-09-18T07:13:20.415 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-18T07:13:20.415 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T07:13:20.479 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T07:13:20.479 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T07:13:20.479 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T07:13:20.479 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T07:13:20.479 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T07:13:20.480 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-18T07:13:20.480 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T07:13:20.480 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T07:13:20.480 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T07:13:20.480 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T07:13:20.512 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-18T07:13:20.514 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-18T07:13:20.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-18T07:13:20.582 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-18T07:13:20.650 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-18T07:13:20.715 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T07:13:20.715 INFO:tasks.ceph:role: osd.2 2024-09-18T07:13:20.716 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-18T07:13:20.716 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T07:13:20.779 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T07:13:20.780 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T07:13:20.780 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T07:13:20.780 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T07:13:20.780 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T07:13:20.780 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-18T07:13:20.780 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T07:13:20.780 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T07:13:20.780 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T07:13:20.780 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T07:13:20.812 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-18T07:13:20.813 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-18T07:13:20.814 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-18T07:13:20.882 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-18T07:13:20.954 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-18T07:13:21.020 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T07:13:21.020 INFO:tasks.ceph:role: osd.3 2024-09-18T07:13:21.020 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-18T07:13:21.021 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T07:13:21.085 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T07:13:21.085 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T07:13:21.085 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T07:13:21.085 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T07:13:21.085 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T07:13:21.085 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-18T07:13:21.086 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T07:13:21.086 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T07:13:21.086 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T07:13:21.086 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T07:13:21.117 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-18T07:13:21.119 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-18T07:13:21.119 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-18T07:13:21.192 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-18T07:13:21.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:21.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:21.363+0000 7f61b791e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-18T07:13:21.366 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:21.363+0000 7f61b791e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-18T07:13:21.366 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:21.363+0000 7f61b791e640 -1 bdev(0x55f7d4dcf800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-18T07:13:21.366 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:21.363+0000 7f61b791e640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-18T07:13:24.535 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-18T07:13:24.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:24.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:24.647+0000 7f8e1b72c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-18T07:13:24.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:24.647+0000 7f8e1b72c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-18T07:13:24.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:24.647+0000 7f8e1b72c640 -1 bdev(0x558750871800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-18T07:13:24.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:24.647+0000 7f8e1b72c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-18T07:13:27.563 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-18T07:13:27.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:27.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:27.679+0000 7f12da7c5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-18T07:13:27.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:27.680+0000 7f12da7c5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-18T07:13:27.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:27.680+0000 7f12da7c5640 -1 bdev(0x5569b0959800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-18T07:13:27.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:27.680+0000 7f12da7c5640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-18T07:13:31.091 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-18T07:13:31.120 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:31.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:31.209+0000 7fc627c1e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-18T07:13:31.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:31.209+0000 7fc627c1e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-18T07:13:31.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:31.209+0000 7fc627c1e640 -1 bdev(0x55ecfaab3800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-18T07:13:31.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:31.209+0000 7fc627c1e640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-18T07:13:34.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-18T07:13:34.671 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-18T07:13:34.702 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T07:13:34.702 INFO:tasks.ceph:role: osd.4 2024-09-18T07:13:34.702 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com 2024-09-18T07:13:34.702 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T07:13:34.778 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T07:13:34.778 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T07:13:34.778 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T07:13:34.778 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T07:13:34.779 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T07:13:34.779 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-09-18T07:13:34.779 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T07:13:34.779 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T07:13:34.779 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T07:13:34.779 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T07:13:35.291 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-09-18T07:13:35.293 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-09-18T07:13:35.293 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-18T07:13:35.431 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-18T07:13:35.464 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-18T07:13:35.528 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T07:13:35.529 INFO:tasks.ceph:role: osd.5 2024-09-18T07:13:35.529 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com 2024-09-18T07:13:35.529 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T07:13:35.602 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T07:13:35.602 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T07:13:35.602 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T07:13:35.602 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T07:13:35.602 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T07:13:35.602 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-09-18T07:13:35.602 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T07:13:35.602 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T07:13:35.602 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T07:13:35.602 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T07:13:36.048 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-09-18T07:13:36.050 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-09-18T07:13:36.051 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-18T07:13:36.142 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-18T07:13:36.171 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-18T07:13:36.236 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T07:13:36.236 INFO:tasks.ceph:role: osd.6 2024-09-18T07:13:36.236 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com 2024-09-18T07:13:36.236 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T07:13:36.308 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T07:13:36.308 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T07:13:36.308 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T07:13:36.308 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T07:13:36.308 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T07:13:36.309 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-09-18T07:13:36.309 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T07:13:36.309 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T07:13:36.309 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T07:13:36.309 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T07:13:36.792 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-09-18T07:13:36.794 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-09-18T07:13:36.795 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-18T07:13:36.929 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-18T07:13:36.962 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-18T07:13:37.029 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T07:13:37.029 INFO:tasks.ceph:role: osd.7 2024-09-18T07:13:37.029 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com 2024-09-18T07:13:37.029 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T07:13:37.103 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T07:13:37.103 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T07:13:37.103 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T07:13:37.103 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T07:13:37.103 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T07:13:37.103 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-09-18T07:13:37.103 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T07:13:37.103 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T07:13:37.103 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T07:13:37.103 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T07:13:37.610 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-09-18T07:13:37.612 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-09-18T07:13:37.612 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-18T07:13:37.750 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-18T07:13:37.786 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:37.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:37.912+0000 7f62e7e15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-18T07:13:37.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:37.912+0000 7f62e7e15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-18T07:13:37.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:37.912+0000 7f62e7e15640 -1 bdev(0x55e72aaed800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-18T07:13:37.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:37.912+0000 7f62e7e15640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-18T07:13:40.854 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-18T07:13:40.884 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:40.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:40.972+0000 7fd2ac644640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-18T07:13:40.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:40.973+0000 7fd2ac644640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-18T07:13:40.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:40.973+0000 7fd2ac644640 -1 bdev(0x55a738d9f800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-18T07:13:40.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:40.973+0000 7fd2ac644640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-18T07:13:44.643 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-18T07:13:44.674 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:44.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:44.762+0000 7f88b821e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-18T07:13:44.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:44.762+0000 7f88b821e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-18T07:13:44.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:44.762+0000 7f88b821e640 -1 bdev(0x5609260df800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-18T07:13:44.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:44.762+0000 7f88b821e640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-18T07:13:47.685 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-18T07:13:47.714 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:47.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:47.804+0000 7fb7cfa1e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-18T07:13:47.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:47.804+0000 7fb7cfa1e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-18T07:13:47.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:47.804+0000 7fb7cfa1e640 -1 bdev(0x558cc4bcf800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-18T07:13:47.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:47.804+0000 7fb7cfa1e640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-18T07:13:50.964 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-18T07:13:50.994 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-18T07:13:50.994 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:50.995 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-18T07:13:51.025 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:51.026 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-18T07:13:51.094 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:51.094 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-18T07:13:51.162 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:51.162 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-18T07:13:51.229 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:51.229 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-18T07:13:51.295 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:51.295 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-18T07:13:51.325 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:51.325 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-18T07:13:51.394 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:51.394 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-18T07:13:51.462 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:51.462 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-18T07:13:51.530 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:51.530 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-18T07:13:51.597 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:51.597 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-18T07:13:51.615 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:51.615 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-18T07:13:51.654 INFO:tasks.ceph:Adding keys to all mons... 2024-09-18T07:13:51.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T07:13:51.658 DEBUG:teuthology.orchestra.run.smithi172:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T07:13:51.697 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.y] 2024-09-18T07:13:51.697 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCPfepmJaWKNRAAth286ZsTT7C0J8EpOERshw== 2024-09-18T07:13:51.697 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.0] 2024-09-18T07:13:51.697 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCRfepml7KxFRAAGpyrk8CKsPth1Ec6+UALvA== 2024-09-18T07:13:51.697 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.1] 2024-09-18T07:13:51.697 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCUfepmhmmnJhAAwmvkSBapCYcIbvuga0bq0w== 2024-09-18T07:13:51.698 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.2] 2024-09-18T07:13:51.698 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCXfepmOD+VKBAAxwAtPQYhVBWiPqxRRQCUzA== 2024-09-18T07:13:51.698 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.3] 2024-09-18T07:13:51.698 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCbfepmu/mJDBAA2CL+q++SY3sqiV7ZfnrM/w== 2024-09-18T07:13:51.698 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.x] 2024-09-18T07:13:51.698 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCPfepmcVhFOhAA25HBYHCoqZim52Tyl9hKLw== 2024-09-18T07:13:51.698 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.4] 2024-09-18T07:13:51.698 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQChfepm3wt3NhAAZxKSwOyEsRv8/FYf2Q1ggQ== 2024-09-18T07:13:51.698 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.5] 2024-09-18T07:13:51.698 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCkfepmzgMEOhAACpsiGBK3C+kBaBQuLWR32Q== 2024-09-18T07:13:51.699 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.6] 2024-09-18T07:13:51.699 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCofepmD+t2LRAAMRIoevkEii+DPsy4VjW6Gg== 2024-09-18T07:13:51.699 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.7] 2024-09-18T07:13:51.699 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCrfepmC+v2LxAAi2yL9VEITpsFz2ThEVl/gA== 2024-09-18T07:13:51.699 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2024-09-18T07:13:51.699 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCQfepmuvRMARAA6an9eI2pN+ZkZDDRcCgBCA== 2024-09-18T07:13:51.699 INFO:teuthology.orchestra.run.smithi070.stdout:[client.1] 2024-09-18T07:13:51.699 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCQfepm95t9BBAAKtBQ5L4S4/xshVm/qSulIg== 2024-09-18T07:13:51.719 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.y] 2024-09-18T07:13:51.719 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCPfepmJaWKNRAAth286ZsTT7C0J8EpOERshw== 2024-09-18T07:13:51.719 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.0] 2024-09-18T07:13:51.719 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCRfepml7KxFRAAGpyrk8CKsPth1Ec6+UALvA== 2024-09-18T07:13:51.719 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.1] 2024-09-18T07:13:51.719 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCUfepmhmmnJhAAwmvkSBapCYcIbvuga0bq0w== 2024-09-18T07:13:51.719 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.2] 2024-09-18T07:13:51.719 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCXfepmOD+VKBAAxwAtPQYhVBWiPqxRRQCUzA== 2024-09-18T07:13:51.720 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.3] 2024-09-18T07:13:51.720 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCbfepmu/mJDBAA2CL+q++SY3sqiV7ZfnrM/w== 2024-09-18T07:13:51.720 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.x] 2024-09-18T07:13:51.720 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCPfepmcVhFOhAA25HBYHCoqZim52Tyl9hKLw== 2024-09-18T07:13:51.720 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.4] 2024-09-18T07:13:51.720 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQChfepm3wt3NhAAZxKSwOyEsRv8/FYf2Q1ggQ== 2024-09-18T07:13:51.720 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.5] 2024-09-18T07:13:51.720 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCkfepmzgMEOhAACpsiGBK3C+kBaBQuLWR32Q== 2024-09-18T07:13:51.720 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.6] 2024-09-18T07:13:51.720 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCofepmD+t2LRAAMRIoevkEii+DPsy4VjW6Gg== 2024-09-18T07:13:51.720 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.7] 2024-09-18T07:13:51.721 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCrfepmC+v2LxAAi2yL9VEITpsFz2ThEVl/gA== 2024-09-18T07:13:51.721 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2024-09-18T07:13:51.721 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCQfepmuvRMARAA6an9eI2pN+ZkZDDRcCgBCA== 2024-09-18T07:13:51.721 INFO:teuthology.orchestra.run.smithi172.stdout:[client.1] 2024-09-18T07:13:51.721 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCQfepm95t9BBAAKtBQ5L4S4/xshVm/qSulIg== 2024-09-18T07:13:51.722 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T07:13:51.743 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T07:13:51.815 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:51.817 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:51.869 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:51.872 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:51.922 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:51.925 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:51.976 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:51.978 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:52.029 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T07:13:52.032 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T07:13:52.083 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:52.085 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:52.136 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:52.138 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:52.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:52.191 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:52.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:52.246 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T07:13:52.298 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T07:13:52.300 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T07:13:52.354 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T07:13:52.357 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T07:13:52.411 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-18T07:13:52.411 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-18T07:13:52.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T07:13:52.531 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-18T07:13:52.555 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-18T07:13:52.620 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T07:13:52.716 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-18T07:13:52.740 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-18T07:13:52.766 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T07:13:52.862 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-18T07:13:52.887 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:52.889 DEBUG:teuthology.orchestra.run.smithi172:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T07:13:52.942 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-18T07:13:52.942 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-18T07:13:52.943 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-18T07:13:52.945 INFO:tasks.ceph.mon.a:Started 2024-09-18T07:13:52.945 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-18T07:13:52.945 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-18T07:13:52.947 INFO:tasks.ceph.mon.c:Started 2024-09-18T07:13:52.947 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-18T07:13:52.948 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-18T07:13:52.984 INFO:tasks.ceph.mon.b:Started 2024-09-18T07:13:52.984 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-18T07:13:52.985 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-18T07:13:52.985 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-18T07:13:52.987 INFO:tasks.ceph.mgr.y:Started 2024-09-18T07:13:52.987 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-18T07:13:52.987 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-18T07:13:52.989 INFO:tasks.ceph.mgr.x:Started 2024-09-18T07:13:52.990 DEBUG:tasks.ceph:set 0 configs 2024-09-18T07:13:52.990 INFO:tasks.ceph:Setting crush tunables to default 2024-09-18T07:13:52.990 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd crush tunables default 2024-09-18T07:13:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.087+0000 7f814138a640 1 Processor -- start 2024-09-18T07:13:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.087+0000 7f814138a640 1 -- start start 2024-09-18T07:13:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.088+0000 7f814138a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f813c0ac320 con 0x7f813c0aae50 2024-09-18T07:13:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.088+0000 7f814138a640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f813c0ac590 con 0x7f813c069050 2024-09-18T07:13:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.088+0000 7f814138a640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f813c0aa650 con 0x7f813c140c40 2024-09-18T07:13:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.088+0000 7f813a7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f813c0aae50 0x7f813c140530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41196/0 (socket says 172.21.15.70:41196) 2024-09-18T07:13:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.088+0000 7f813b7fe640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f813c140c40 0x7f813c14ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:50290/0 (socket says 172.21.15.70:50290) 2024-09-18T07:13:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.088+0000 7f813a7fc640 1 -- 172.21.15.70:0/2612138883 learned_addr learned my addr 172.21.15.70:0/2612138883 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:13:53.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.244+0000 7f8139ffb640 1 -- 172.21.15.70:0/2612138883 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3497219899 0 0) 0x7f813c0ac320 con 0x7f813c0aae50 2024-09-18T07:13:53.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.244+0000 7f8139ffb640 1 -- 172.21.15.70:0/2612138883 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8124003520 con 0x7f813c0aae50 2024-09-18T07:13:53.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.244+0000 7f8139ffb640 1 -- 172.21.15.70:0/2612138883 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 91610767 0 0) 0x7f813c0aa650 con 0x7f813c140c40 2024-09-18T07:13:53.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.244+0000 7f8139ffb640 1 -- 172.21.15.70:0/2612138883 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f813c0ac320 con 0x7f813c140c40 2024-09-18T07:13:53.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.244+0000 7f8139ffb640 1 -- 172.21.15.70:0/2612138883 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 374287741 0 0) 0x7f813c0ac320 con 0x7f813c140c40 2024-09-18T07:13:53.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.244+0000 7f8139ffb640 1 -- 172.21.15.70:0/2612138883 >> v1:172.21.15.172:6789/0 conn(0x7f813c069050 legacy=0x7f813c143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:53.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.244+0000 7f8139ffb640 1 -- 172.21.15.70:0/2612138883 >> v1:172.21.15.70:6789/0 conn(0x7f813c0aae50 legacy=0x7f813c140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:53.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.244+0000 7f8139ffb640 1 -- 172.21.15.70:0/2612138883 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813c0aa930 con 0x7f813c140c40 2024-09-18T07:13:53.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.244+0000 7f8139ffb640 1 -- 172.21.15.70:0/2612138883 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f81340019f0 con 0x7f813c140c40 2024-09-18T07:13:53.248 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.245+0000 7f8139ffb640 1 -- 172.21.15.70:0/2612138883 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8134001c10 con 0x7f813c140c40 2024-09-18T07:13:53.248 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.245+0000 7f8139ffb640 1 -- 172.21.15.70:0/2612138883 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8134001fa0 con 0x7f813c140c40 2024-09-18T07:13:53.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.245+0000 7f814138a640 1 -- 172.21.15.70:0/2612138883 >> v1:172.21.15.70:6790/0 conn(0x7f813c140c40 legacy=0x7f813c14ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:53.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.245+0000 7f814138a640 1 -- 172.21.15.70:0/2612138883 shutdown_connections 2024-09-18T07:13:53.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.245+0000 7f814138a640 1 -- 172.21.15.70:0/2612138883 >> 172.21.15.70:0/2612138883 conn(0x7f813c0b9370 msgr2=0x7f813c0b9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:53.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.245+0000 7f814138a640 1 -- 172.21.15.70:0/2612138883 shutdown_connections 2024-09-18T07:13:53.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.245+0000 7f814138a640 1 -- 172.21.15.70:0/2612138883 wait complete. 2024-09-18T07:13:53.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.247+0000 7f814138a640 1 Processor -- start 2024-09-18T07:13:53.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.247+0000 7f814138a640 1 -- start start 2024-09-18T07:13:53.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.248+0000 7f813affd640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f813c069050 0x7f813c0b0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:50326/0 (socket says 172.21.15.70:50326) 2024-09-18T07:13:53.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.248+0000 7f813affd640 1 -- 172.21.15.70:0/3857264092 learned_addr learned my addr 172.21.15.70:0/3857264092 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:13:53.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.248+0000 7f814138a640 1 -- 172.21.15.70:0/3857264092 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f813c0b2650 con 0x7f813c140c40 2024-09-18T07:13:53.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.248+0000 7f814138a640 1 -- 172.21.15.70:0/3857264092 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f813c1dbfd0 con 0x7f813c0aae50 2024-09-18T07:13:53.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.248+0000 7f814138a640 1 -- 172.21.15.70:0/3857264092 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f813c1500d0 con 0x7f813c069050 2024-09-18T07:13:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.248+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1617918400 0 0) 0x7f813c0b2650 con 0x7f813c140c40 2024-09-18T07:13:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.248+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8128003550 con 0x7f813c140c40 2024-09-18T07:13:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.248+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1682397721 0 0) 0x7f8128003550 con 0x7f813c140c40 2024-09-18T07:13:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.248+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f813c0b2650 con 0x7f813c140c40 2024-09-18T07:13:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.248+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8134001f60 con 0x7f813c140c40 2024-09-18T07:13:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.248+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1840293820 0 0) 0x7f813c0b2650 con 0x7f813c140c40 2024-09-18T07:13:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.249+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 >> v1:172.21.15.70:6790/0 conn(0x7f813c069050 legacy=0x7f813c0b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.249+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 >> v1:172.21.15.172:6789/0 conn(0x7f813c0aae50 legacy=0x7f813c1db8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.249+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813c0b2a70 con 0x7f813c140c40 2024-09-18T07:13:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.249+0000 7f814138a640 1 -- 172.21.15.70:0/3857264092 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f813c0b1160 con 0x7f813c140c40 2024-09-18T07:13:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.249+0000 7f814138a640 1 -- 172.21.15.70:0/3857264092 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f813c1e4580 con 0x7f813c140c40 2024-09-18T07:13:53.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.250+0000 7f814138a640 1 -- 172.21.15.70:0/3857264092 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f813c0b2d00 con 0x7f813c140c40 2024-09-18T07:13:53.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.251+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8134001b70 con 0x7f813c140c40 2024-09-18T07:13:53.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.251+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8134004fb0 con 0x7f813c140c40 2024-09-18T07:13:53.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.251+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f81340054e0 con 0x7f813c140c40 2024-09-18T07:13:53.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.251+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 85744489 0 0) 0x7f81340069e0 con 0x7f813c140c40 2024-09-18T07:13:53.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.253+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7f81340057d0 con 0x7f813c140c40 2024-09-18T07:13:53.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.320+0000 7f814138a640 1 -- 172.21.15.70:0/3857264092 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f813c0b76b0 con 0x7f813c140c40 2024-09-18T07:13:53.324 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.321+0000 7f81237fe640 1 -- 172.21.15.70:0/3857264092 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (unknown 2277712955 0 0) 0x7f8134005a20 con 0x7f813c140c40 2024-09-18T07:13:53.324 INFO:teuthology.orchestra.run.smithi070.stderr:adjusted tunables profile to default 2024-09-18T07:13:53.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.322+0000 7f814138a640 1 -- 172.21.15.70:0/3857264092 >> v1:172.21.15.70:6789/0 conn(0x7f813c140c40 legacy=0x7f813c1e2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:53.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.323+0000 7f814138a640 1 -- 172.21.15.70:0/3857264092 shutdown_connections 2024-09-18T07:13:53.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.323+0000 7f814138a640 1 -- 172.21.15.70:0/3857264092 >> 172.21.15.70:0/3857264092 conn(0x7f813c0b9370 msgr2=0x7f813c0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:53.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.323+0000 7f814138a640 1 -- 172.21.15.70:0/3857264092 shutdown_connections 2024-09-18T07:13:53.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:53.323+0000 7f814138a640 1 -- 172.21.15.70:0/3857264092 wait complete. 2024-09-18T07:13:53.339 INFO:tasks.ceph:check_enable_crimson: False 2024-09-18T07:13:53.339 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-18T07:13:53.340 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:53.340 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-18T07:13:53.364 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:53.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-18T07:13:53.429 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:53.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-18T07:13:53.494 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:13:53.494 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-18T07:13:53.559 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:53.559 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-18T07:13:53.584 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:53.584 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-18T07:13:53.650 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:53.650 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-18T07:13:53.714 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-18T07:13:53.714 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-18T07:13:53.780 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new a4a2c5fa-37b8-4776-bf81-f142248da0da 0 2024-09-18T07:13:53.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.917+0000 7f94fbb3d640 1 Processor -- start 2024-09-18T07:13:53.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.917+0000 7f94fbb3d640 1 -- start start 2024-09-18T07:13:53.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.918+0000 7f94fbb3d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f40b3670 con 0x7f94f4143650 2024-09-18T07:13:53.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.918+0000 7f94fbb3d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f4143ca0 con 0x7f94f40b61b0 2024-09-18T07:13:53.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.918+0000 7f94fbb3d640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f40b57f0 con 0x7f94f40aab50 2024-09-18T07:13:53.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.918+0000 7f94f98d9640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f94f40b61b0 0x7f94f40b2f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55634/0 (socket says 172.21.15.172:55634) 2024-09-18T07:13:53.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.918+0000 7f94f98d9640 1 -- 172.21.15.172:0/2751974228 learned_addr learned my addr 172.21.15.172:0/2751974228 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:53.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.919+0000 7f94f88d7640 1 -- 172.21.15.172:0/2751974228 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 839803251 0 0) 0x7f94f4143ca0 con 0x7f94f40b61b0 2024-09-18T07:13:53.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.919+0000 7f94f88d7640 1 -- 172.21.15.172:0/2751974228 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94e0003520 con 0x7f94f40b61b0 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.919+0000 7f94f88d7640 1 -- 172.21.15.172:0/2751974228 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1551464285 0 0) 0x7f94e8001110 con 0x7f94f4143650 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.919+0000 7f94f88d7640 1 -- 172.21.15.172:0/2751974228 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94f4143ca0 con 0x7f94f4143650 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.919+0000 7f94f88d7640 1 -- 172.21.15.172:0/2751974228 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2179223602 0 0) 0x7f94f40b57f0 con 0x7f94f40aab50 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.919+0000 7f94f88d7640 1 -- 172.21.15.172:0/2751974228 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94e8001110 con 0x7f94f40aab50 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.919+0000 7f94f88d7640 1 -- 172.21.15.172:0/2751974228 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1688809959 0 0) 0x7f94e0003520 con 0x7f94f40b61b0 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.919+0000 7f94f88d7640 1 -- 172.21.15.172:0/2751974228 >> v1:172.21.15.70:6790/0 conn(0x7f94f40aab50 legacy=0x7f94f414ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.919+0000 7f94f88d7640 1 -- 172.21.15.172:0/2751974228 >> v1:172.21.15.70:6789/0 conn(0x7f94f4143650 legacy=0x7f94f40aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.919+0000 7f94f88d7640 1 -- 172.21.15.172:0/2751974228 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f4140cb0 con 0x7f94f40b61b0 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.919+0000 7f94f88d7640 1 -- 172.21.15.172:0/2751974228 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f94f0001690 con 0x7f94f40b61b0 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.919+0000 7f94f88d7640 1 -- 172.21.15.172:0/2751974228 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f94f00018b0 con 0x7f94f40b61b0 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.919+0000 7f94f88d7640 1 -- 172.21.15.172:0/2751974228 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f94f0001d00 con 0x7f94f40b61b0 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.920+0000 7f94fbb3d640 1 -- 172.21.15.172:0/2751974228 >> v1:172.21.15.172:6789/0 conn(0x7f94f40b61b0 legacy=0x7f94f40b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.920+0000 7f94fbb3d640 1 -- 172.21.15.172:0/2751974228 shutdown_connections 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.920+0000 7f94fbb3d640 1 -- 172.21.15.172:0/2751974228 >> 172.21.15.172:0/2751974228 conn(0x7f94f4068cd0 msgr2=0x7f94f40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:53.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.920+0000 7f94fbb3d640 1 -- 172.21.15.172:0/2751974228 shutdown_connections 2024-09-18T07:13:53.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.920+0000 7f94fbb3d640 1 -- 172.21.15.172:0/2751974228 wait complete. 2024-09-18T07:13:53.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.921+0000 7f94fbb3d640 1 Processor -- start 2024-09-18T07:13:53.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.921+0000 7f94fbb3d640 1 -- start start 2024-09-18T07:13:53.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.922+0000 7f94fbb3d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f413b700 con 0x7f94f4143650 2024-09-18T07:13:53.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.922+0000 7f94fbb3d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f413b8d0 con 0x7f94f40b61b0 2024-09-18T07:13:53.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.922+0000 7f94fbb3d640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f413f8e0 con 0x7f94f40aab50 2024-09-18T07:13:53.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.922+0000 7f94f90d8640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f94f40b61b0 0x7f94f41dc160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55640/0 (socket says 172.21.15.172:55640) 2024-09-18T07:13:53.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.922+0000 7f94f90d8640 1 -- 172.21.15.172:0/3622237765 learned_addr learned my addr 172.21.15.172:0/3622237765 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:53.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1864075777 0 0) 0x7f94f413b8d0 con 0x7f94f40b61b0 2024-09-18T07:13:53.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94d8003520 con 0x7f94f40b61b0 2024-09-18T07:13:53.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 245422018 0 0) 0x7f94f413f8e0 con 0x7f94f40aab50 2024-09-18T07:13:53.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94f413b8d0 con 0x7f94f40aab50 2024-09-18T07:13:53.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1548694076 0 0) 0x7f94f413b700 con 0x7f94f4143650 2024-09-18T07:13:53.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94f413f8e0 con 0x7f94f4143650 2024-09-18T07:13:53.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1135960694 0 0) 0x7f94d8003520 con 0x7f94f40b61b0 2024-09-18T07:13:53.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f94f413b700 con 0x7f94f40b61b0 2024-09-18T07:13:53.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f94e8001a50 con 0x7f94f40b61b0 2024-09-18T07:13:53.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2860989545 0 0) 0x7f94f413b700 con 0x7f94f40b61b0 2024-09-18T07:13:53.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 >> v1:172.21.15.70:6790/0 conn(0x7f94f40aab50 legacy=0x7f94f41dba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:53.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 >> v1:172.21.15.70:6789/0 conn(0x7f94f4143650 legacy=0x7f94f41e3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:53.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f413fab0 con 0x7f94f40b61b0 2024-09-18T07:13:53.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94fbb3d640 1 -- 172.21.15.172:0/3622237765 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f94f413c1a0 con 0x7f94f40b61b0 2024-09-18T07:13:53.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94fbb3d640 1 -- 172.21.15.172:0/3622237765 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f94f41e4780 con 0x7f94f40b61b0 2024-09-18T07:13:53.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f94e8001cb0 con 0x7f94f40b61b0 2024-09-18T07:13:53.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f94e8003a60 con 0x7f94f40b61b0 2024-09-18T07:13:53.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f94e8004050 con 0x7f94f40b61b0 2024-09-18T07:13:53.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.923+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (unknown 1689787647 0 0) 0x7f94e80066d0 con 0x7f94f40b61b0 2024-09-18T07:13:53.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.924+0000 7f94fbb3d640 1 -- 172.21.15.172:0/3622237765 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94dc005080 con 0x7f94f40b61b0 2024-09-18T07:13:53.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.926+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7f94f40b3670 con 0x7f94f40b61b0 2024-09-18T07:13:53.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:53.995+0000 7f94fbb3d640 1 -- 172.21.15.172:0/3622237765 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd new", "uuid": "a4a2c5fa-37b8-4776-bf81-f142248da0da", "id": 0} v 0) -- 0x7f94dc005370 con 0x7f94f40b61b0 2024-09-18T07:13:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.001+0000 7f94e67fc640 1 -- 172.21.15.172:0/3622237765 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a4a2c5fa-37b8-4776-bf81-f142248da0da", "id": 0}]=0 v3) ==== 112+0+2 (unknown 3477242869 0 732707515) 0x7f94e8006950 con 0x7f94f40b61b0 2024-09-18T07:13:54.003 INFO:teuthology.orchestra.run.smithi172.stdout:0 2024-09-18T07:13:54.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.001+0000 7f94fbb3d640 1 -- 172.21.15.172:0/3622237765 >> v1:172.21.15.172:6789/0 conn(0x7f94f40b61b0 legacy=0x7f94f41dc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.001+0000 7f94fbb3d640 1 -- 172.21.15.172:0/3622237765 shutdown_connections 2024-09-18T07:13:54.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.001+0000 7f94fbb3d640 1 -- 172.21.15.172:0/3622237765 >> 172.21.15.172:0/3622237765 conn(0x7f94f4068cd0 msgr2=0x7f94f4133780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:54.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.002+0000 7f94fbb3d640 1 -- 172.21.15.172:0/3622237765 shutdown_connections 2024-09-18T07:13:54.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.002+0000 7f94fbb3d640 1 -- 172.21.15.172:0/3622237765 wait complete. 2024-09-18T07:13:54.014 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 3fab2225-9881-4caf-8142-41b857b9b99a 1 2024-09-18T07:13:54.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.110+0000 7f38d2885640 1 Processor -- start 2024-09-18T07:13:54.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.110+0000 7f38d2885640 1 -- start start 2024-09-18T07:13:54.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38d2885640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38cc0ac320 con 0x7f38cc0aae50 2024-09-18T07:13:54.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38d2885640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38cc0ac590 con 0x7f38cc069050 2024-09-18T07:13:54.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38d2885640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38cc0aa650 con 0x7f38cc140c40 2024-09-18T07:13:54.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38cbfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f38cc069050 0x7f38cc143390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55646/0 (socket says 172.21.15.172:55646) 2024-09-18T07:13:54.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38cbfff640 1 -- 172.21.15.172:0/3364819535 learned_addr learned my addr 172.21.15.172:0/3364819535 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:54.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38caffd640 1 -- 172.21.15.172:0/3364819535 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2924237303 0 0) 0x7f38cc0ac590 con 0x7f38cc069050 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38d0e22640 1 -- 172.21.15.172:0/3364819535 >> v1:172.21.15.70:6790/0 conn(0x7f38cc140c40 legacy=0x7f38cc14ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38d0e22640 1 -- 172.21.15.172:0/3364819535 >> v1:172.21.15.70:6790/0 conn(0x7f38cc140c40 legacy=0x7f38cc14ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38d0e22640 1 --1- 172.21.15.172:0/3364819535 >> v1:172.21.15.70:6790/0 conn(0x7f38cc140c40 0x7f38cc14ec70 :-1 s=OPENED pgs=20 cs=1 l=1).handle_message read tag failed 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38caffd640 1 -- 172.21.15.172:0/3364819535 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38b4003520 con 0x7f38cc069050 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38d0e22640 1 --1- 172.21.15.172:0/3364819535 >> v1:172.21.15.70:6790/0 conn(0x7f38cc140c40 0x7f38cc14ec70 :-1 s=OPENED pgs=20 cs=1 l=1).fault on lossy channel, failing 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38caffd640 1 -- 172.21.15.172:0/3364819535 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 816635638 0 0) 0x7f38cc0ac320 con 0x7f38cc0aae50 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38caffd640 1 -- 172.21.15.172:0/3364819535 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38cc0ac590 con 0x7f38cc0aae50 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38caffd640 1 -- 172.21.15.172:0/3364819535 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2723022960 0 0) 0x7f38b4003520 con 0x7f38cc069050 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38caffd640 1 -- 172.21.15.172:0/3364819535 >> v1:172.21.15.70:6790/0 conn(0x7f38cc140c40 legacy=0x7f38cc14ec70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38caffd640 1 -- 172.21.15.172:0/3364819535 >> v1:172.21.15.70:6789/0 conn(0x7f38cc0aae50 legacy=0x7f38cc140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38caffd640 1 -- 172.21.15.172:0/3364819535 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc0aa930 con 0x7f38cc069050 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38caffd640 1 -- 172.21.15.172:0/3364819535 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f38bc001690 con 0x7f38cc069050 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38caffd640 1 -- 172.21.15.172:0/3364819535 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f38bc0018b0 con 0x7f38cc069050 2024-09-18T07:13:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.111+0000 7f38caffd640 1 -- 172.21.15.172:0/3364819535 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f38bc001c80 con 0x7f38cc069050 2024-09-18T07:13:54.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.112+0000 7f38d2885640 1 -- 172.21.15.172:0/3364819535 >> v1:172.21.15.172:6789/0 conn(0x7f38cc069050 legacy=0x7f38cc143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.112+0000 7f38d2885640 1 -- 172.21.15.172:0/3364819535 shutdown_connections 2024-09-18T07:13:54.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.112+0000 7f38d2885640 1 -- 172.21.15.172:0/3364819535 >> 172.21.15.172:0/3364819535 conn(0x7f38cc0b9370 msgr2=0x7f38cc0b9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:54.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.112+0000 7f38d2885640 1 -- 172.21.15.172:0/3364819535 shutdown_connections 2024-09-18T07:13:54.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.112+0000 7f38d2885640 1 -- 172.21.15.172:0/3364819535 wait complete. 2024-09-18T07:13:54.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.113+0000 7f38d2885640 1 Processor -- start 2024-09-18T07:13:54.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.113+0000 7f38d2885640 1 -- start start 2024-09-18T07:13:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38d2885640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38cc13afa0 con 0x7f38cc0aae50 2024-09-18T07:13:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38d2885640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38cc1391c0 con 0x7f38cc069050 2024-09-18T07:13:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38d2885640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38cc1d5130 con 0x7f38cc140c40 2024-09-18T07:13:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38cbfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f38cc069050 0x7f38cc1dc130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55652/0 (socket says 172.21.15.172:55652) 2024-09-18T07:13:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38cbfff640 1 -- 172.21.15.172:0/1202930878 learned_addr learned my addr 172.21.15.172:0/1202930878 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 964238331 0 0) 0x7f38cc1391c0 con 0x7f38cc069050 2024-09-18T07:13:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38b8003550 con 0x7f38cc069050 2024-09-18T07:13:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1933960629 0 0) 0x7f38cc13afa0 con 0x7f38cc0aae50 2024-09-18T07:13:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38cc1391c0 con 0x7f38cc0aae50 2024-09-18T07:13:54.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4280369505 0 0) 0x7f38cc1d5130 con 0x7f38cc140c40 2024-09-18T07:13:54.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38cc13afa0 con 0x7f38cc140c40 2024-09-18T07:13:54.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2832903624 0 0) 0x7f38b8003550 con 0x7f38cc069050 2024-09-18T07:13:54.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f38cc1d5130 con 0x7f38cc069050 2024-09-18T07:13:54.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f38bc001c00 con 0x7f38cc069050 2024-09-18T07:13:54.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 627635766 0 0) 0x7f38cc1391c0 con 0x7f38cc0aae50 2024-09-18T07:13:54.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f38b8003550 con 0x7f38cc0aae50 2024-09-18T07:13:54.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f38c0001870 con 0x7f38cc0aae50 2024-09-18T07:13:54.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1954471138 0 0) 0x7f38cc13afa0 con 0x7f38cc140c40 2024-09-18T07:13:54.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f38cc1391c0 con 0x7f38cc140c40 2024-09-18T07:13:54.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.114+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f38c4001a90 con 0x7f38cc140c40 2024-09-18T07:13:54.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.115+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1564105054 0 0) 0x7f38cc1d5130 con 0x7f38cc069050 2024-09-18T07:13:54.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.115+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 >> v1:172.21.15.70:6790/0 conn(0x7f38cc140c40 legacy=0x7f38cc1e3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.115+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 >> v1:172.21.15.70:6789/0 conn(0x7f38cc0aae50 legacy=0x7f38cc1d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.115+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc1d5300 con 0x7f38cc069050 2024-09-18T07:13:54.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.115+0000 7f38d2885640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f38cc1dc840 con 0x7f38cc069050 2024-09-18T07:13:54.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.115+0000 7f38d2885640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f38cc139cb0 con 0x7f38cc069050 2024-09-18T07:13:54.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.115+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f38bc004bf0 con 0x7f38cc069050 2024-09-18T07:13:54.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.115+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f38bc005080 con 0x7f38cc069050 2024-09-18T07:13:54.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.115+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f38bc005630 con 0x7f38cc069050 2024-09-18T07:13:54.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.115+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (unknown 2568610696 0 0) 0x7f38bc006d70 con 0x7f38cc069050 2024-09-18T07:13:54.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.116+0000 7f38d2885640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ac005080 con 0x7f38cc069050 2024-09-18T07:13:54.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.117+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7f38bc001880 con 0x7f38cc069050 2024-09-18T07:13:54.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.188+0000 7f38d2885640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd new", "uuid": "3fab2225-9881-4caf-8142-41b857b9b99a", "id": 1} v 0) -- 0x7f38ac005370 con 0x7f38cc069050 2024-09-18T07:13:54.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.188+0000 7f38cbfff640 1 -- 172.21.15.172:0/1202930878 >> v1:172.21.15.172:6789/0 conn(0x7f38cc069050 legacy=0x7f38cc1dc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-18T07:13:54.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.188+0000 7f38cbfff640 1 -- 172.21.15.172:0/1202930878 >> v1:172.21.15.172:6789/0 conn(0x7f38cc069050 legacy=0x7f38cc1dc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T07:13:54.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.188+0000 7f38cbfff640 1 --1- 172.21.15.172:0/1202930878 >> v1:172.21.15.172:6789/0 conn(0x7f38cc069050 0x7f38cc1dc130 :-1 s=OPENED pgs=15 cs=1 l=1).handle_message read tag failed 2024-09-18T07:13:54.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.188+0000 7f38cbfff640 1 --1- 172.21.15.172:0/1202930878 >> v1:172.21.15.172:6789/0 conn(0x7f38cc069050 0x7f38cc1dc130 :-1 s=OPENED pgs=15 cs=1 l=1).fault on lossy channel, failing 2024-09-18T07:13:54.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.188+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 >> v1:172.21.15.172:6789/0 conn(0x7f38cc069050 legacy=0x7f38cc1dc130 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T07:13:54.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.188+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f38cc1d5130 con 0x7f38cc0aae50 2024-09-18T07:13:54.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.188+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f38b8003550 con 0x7f38b800a9c0 2024-09-18T07:13:54.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.188+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f38cc13afa0 con 0x7f38cc140c40 2024-09-18T07:13:54.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.188+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2834884095 0 0) 0x7f38b8003550 con 0x7f38b800a9c0 2024-09-18T07:13:54.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.188+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.172:6789/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f38b8013580 con 0x7f38b800a9c0 2024-09-18T07:13:54.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.189+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4203873552 0 0) 0x7f38cc13afa0 con 0x7f38cc140c40 2024-09-18T07:13:54.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.189+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.70:6790/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f38b8003550 con 0x7f38cc140c40 2024-09-18T07:13:54.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.189+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 87529396 0 0) 0x7f38cc1d5130 con 0x7f38cc0aae50 2024-09-18T07:13:54.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.189+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.70:6789/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f38cc13afa0 con 0x7f38cc0aae50 2024-09-18T07:13:54.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.189+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (unknown 608851519 0 0) 0x7f38b8013580 con 0x7f38b800a9c0 2024-09-18T07:13:54.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.189+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 >> v1:172.21.15.70:6790/0 conn(0x7f38cc140c40 legacy=0x7f38b80090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.189+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 >> v1:172.21.15.70:6789/0 conn(0x7f38cc0aae50 legacy=0x7f38b80109f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.189+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) -- 0x7f38b80156d0 con 0x7f38b800a9c0 2024-09-18T07:13:54.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.189+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd new", "uuid": "3fab2225-9881-4caf-8142-41b857b9b99a", "id": 1} v 0) -- 0x7f38cc1d5130 con 0x7f38b800a9c0 2024-09-18T07:13:54.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.189+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f38c00037b0 con 0x7f38b800a9c0 2024-09-18T07:13:54.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.189+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f38c00035f0 con 0x7f38b800a9c0 2024-09-18T07:13:54.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.189+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f38c0003b30 con 0x7f38b800a9c0 2024-09-18T07:13:54.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.196+0000 7f38c8ff9640 1 -- 172.21.15.172:0/1202930878 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3fab2225-9881-4caf-8142-41b857b9b99a", "id": 1}]=0 v4) ==== 112+0+2 (unknown 3521803671 0 940483276) 0x7f38c0003e20 con 0x7f38b800a9c0 2024-09-18T07:13:54.198 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-09-18T07:13:54.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.197+0000 7f38d2885640 1 -- 172.21.15.172:0/1202930878 >> v1:172.21.15.172:6789/0 conn(0x7f38b800a9c0 legacy=0x7f38b800ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.197+0000 7f38d2885640 1 -- 172.21.15.172:0/1202930878 shutdown_connections 2024-09-18T07:13:54.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.197+0000 7f38d2885640 1 -- 172.21.15.172:0/1202930878 >> 172.21.15.172:0/1202930878 conn(0x7f38cc0b9370 msgr2=0x7f38cc0b6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:54.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.197+0000 7f38d2885640 1 -- 172.21.15.172:0/1202930878 shutdown_connections 2024-09-18T07:13:54.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.197+0000 7f38d2885640 1 -- 172.21.15.172:0/1202930878 wait complete. 2024-09-18T07:13:54.210 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 5d6764cd-e7a4-45df-9cfb-88d9e9a2675f 2 2024-09-18T07:13:54.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.307+0000 7f3e26a16640 1 Processor -- start 2024-09-18T07:13:54.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.307+0000 7f3e26a16640 1 -- start start 2024-09-18T07:13:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.308+0000 7f3e26a16640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e200b2520 con 0x7f3e20143980 2024-09-18T07:13:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.308+0000 7f3e26a16640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e200b0ff0 con 0x7f3e20068b20 2024-09-18T07:13:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.308+0000 7f3e26a16640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e200b0180 con 0x7f3e200b1510 2024-09-18T07:13:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.308+0000 7f3e1ffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3e20068b20 0x7f3e20143270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55662/0 (socket says 172.21.15.172:55662) 2024-09-18T07:13:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.308+0000 7f3e1ffff640 1 -- 172.21.15.172:0/2862970463 learned_addr learned my addr 172.21.15.172:0/2862970463 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e1f7fe640 1 -- 172.21.15.172:0/2862970463 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 640979806 0 0) 0x7f3e200b0ff0 con 0x7f3e20068b20 2024-09-18T07:13:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e1f7fe640 1 -- 172.21.15.172:0/2862970463 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e0c003520 con 0x7f3e20068b20 2024-09-18T07:13:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e1f7fe640 1 -- 172.21.15.172:0/2862970463 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 855816013 0 0) 0x7f3e200b2520 con 0x7f3e20143980 2024-09-18T07:13:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e1f7fe640 1 -- 172.21.15.172:0/2862970463 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e200b0ff0 con 0x7f3e20143980 2024-09-18T07:13:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e1f7fe640 1 -- 172.21.15.172:0/2862970463 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1982456312 0 0) 0x7f3e200b0180 con 0x7f3e200b1510 2024-09-18T07:13:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e1f7fe640 1 -- 172.21.15.172:0/2862970463 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e200b2520 con 0x7f3e200b1510 2024-09-18T07:13:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e1f7fe640 1 -- 172.21.15.172:0/2862970463 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 942522682 0 0) 0x7f3e0c003520 con 0x7f3e20068b20 2024-09-18T07:13:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e1f7fe640 1 -- 172.21.15.172:0/2862970463 >> v1:172.21.15.70:6790/0 conn(0x7f3e200b1510 legacy=0x7f3e2014eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e1f7fe640 1 -- 172.21.15.172:0/2862970463 >> v1:172.21.15.70:6789/0 conn(0x7f3e20143980 legacy=0x7f3e20140490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e1f7fe640 1 -- 172.21.15.172:0/2862970463 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e20140ba0 con 0x7f3e20068b20 2024-09-18T07:13:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e1f7fe640 1 -- 172.21.15.172:0/2862970463 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f3e10001690 con 0x7f3e20068b20 2024-09-18T07:13:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e1f7fe640 1 -- 172.21.15.172:0/2862970463 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3e100018b0 con 0x7f3e20068b20 2024-09-18T07:13:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e1f7fe640 1 -- 172.21.15.172:0/2862970463 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f3e10001d00 con 0x7f3e20068b20 2024-09-18T07:13:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.309+0000 7f3e26a16640 1 -- 172.21.15.172:0/2862970463 >> v1:172.21.15.172:6789/0 conn(0x7f3e20068b20 legacy=0x7f3e20143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.310+0000 7f3e26a16640 1 -- 172.21.15.172:0/2862970463 shutdown_connections 2024-09-18T07:13:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.310+0000 7f3e26a16640 1 -- 172.21.15.172:0/2862970463 >> 172.21.15.172:0/2862970463 conn(0x7f3e2006cd30 msgr2=0x7f3e2006d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:54.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.310+0000 7f3e26a16640 1 -- 172.21.15.172:0/2862970463 shutdown_connections 2024-09-18T07:13:54.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.310+0000 7f3e26a16640 1 -- 172.21.15.172:0/2862970463 wait complete. 2024-09-18T07:13:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.311+0000 7f3e26a16640 1 Processor -- start 2024-09-18T07:13:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.311+0000 7f3e26a16640 1 -- start start 2024-09-18T07:13:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.312+0000 7f3e26a16640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e20146c60 con 0x7f3e20068b20 2024-09-18T07:13:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.312+0000 7f3e26a16640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e20146f40 con 0x7f3e20143980 2024-09-18T07:13:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.312+0000 7f3e26a16640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e20144870 con 0x7f3e200b1510 2024-09-18T07:13:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.312+0000 7f3e24fb3640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3e20143980 0x7f3e20131dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55672/0 (socket says 172.21.15.172:55672) 2024-09-18T07:13:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.312+0000 7f3e24fb3640 1 -- 172.21.15.172:0/1328793765 learned_addr learned my addr 172.21.15.172:0/1328793765 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1576965662 0 0) 0x7f3e20146f40 con 0x7f3e20143980 2024-09-18T07:13:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e00003520 con 0x7f3e20143980 2024-09-18T07:13:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3400273333 0 0) 0x7f3e20146c60 con 0x7f3e20068b20 2024-09-18T07:13:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e20146f40 con 0x7f3e20068b20 2024-09-18T07:13:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 764332849 0 0) 0x7f3e20144870 con 0x7f3e200b1510 2024-09-18T07:13:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e20146c60 con 0x7f3e200b1510 2024-09-18T07:13:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2920249915 0 0) 0x7f3e00003520 con 0x7f3e20143980 2024-09-18T07:13:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3e20144870 con 0x7f3e20143980 2024-09-18T07:13:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f3e18001c80 con 0x7f3e20143980 2024-09-18T07:13:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 52616108 0 0) 0x7f3e20146f40 con 0x7f3e20068b20 2024-09-18T07:13:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3e00003520 con 0x7f3e20068b20 2024-09-18T07:13:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f3e10001c00 con 0x7f3e20068b20 2024-09-18T07:13:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 720909638 0 0) 0x7f3e20146c60 con 0x7f3e200b1510 2024-09-18T07:13:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3e20146f40 con 0x7f3e200b1510 2024-09-18T07:13:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f3e08001730 con 0x7f3e200b1510 2024-09-18T07:13:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3478492969 0 0) 0x7f3e20144870 con 0x7f3e20143980 2024-09-18T07:13:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 >> v1:172.21.15.70:6790/0 conn(0x7f3e200b1510 legacy=0x7f3e20142080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 >> v1:172.21.15.70:6789/0 conn(0x7f3e20068b20 legacy=0x7f3e201dc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e201453a0 con 0x7f3e20143980 2024-09-18T07:13:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e26a16640 1 -- 172.21.15.172:0/1328793765 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3e20145c90 con 0x7f3e20143980 2024-09-18T07:13:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e26a16640 1 -- 172.21.15.172:0/1328793765 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f3e201dcd70 con 0x7f3e20143980 2024-09-18T07:13:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3e18003990 con 0x7f3e20143980 2024-09-18T07:13:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f3e18003d60 con 0x7f3e20143980 2024-09-18T07:13:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.313+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f3e18004350 con 0x7f3e20143980 2024-09-18T07:13:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.314+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (unknown 1961347761 0 0) 0x7f3e18005f60 con 0x7f3e20143980 2024-09-18T07:13:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.314+0000 7f3e26a16640 1 -- 172.21.15.172:0/1328793765 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e04005080 con 0x7f3e20143980 2024-09-18T07:13:54.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.316+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7f3e18003d60 con 0x7f3e20143980 2024-09-18T07:13:54.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.386+0000 7f3e26a16640 1 -- 172.21.15.172:0/1328793765 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd new", "uuid": "5d6764cd-e7a4-45df-9cfb-88d9e9a2675f", "id": 2} v 0) -- 0x7f3e04005370 con 0x7f3e20143980 2024-09-18T07:13:54.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.394+0000 7f3e1d7fa640 1 -- 172.21.15.172:0/1328793765 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5d6764cd-e7a4-45df-9cfb-88d9e9a2675f", "id": 2}]=0 v5) ==== 112+0+2 (unknown 1161096078 0 216599125) 0x7f3e18004640 con 0x7f3e20143980 2024-09-18T07:13:54.396 INFO:teuthology.orchestra.run.smithi172.stdout:2 2024-09-18T07:13:54.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.395+0000 7f3e26a16640 1 -- 172.21.15.172:0/1328793765 >> v1:172.21.15.172:6789/0 conn(0x7f3e20143980 legacy=0x7f3e20131dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.395+0000 7f3e26a16640 1 -- 172.21.15.172:0/1328793765 shutdown_connections 2024-09-18T07:13:54.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.395+0000 7f3e26a16640 1 -- 172.21.15.172:0/1328793765 >> 172.21.15.172:0/1328793765 conn(0x7f3e2006cd30 msgr2=0x7f3e2013c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:54.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.395+0000 7f3e26a16640 1 -- 172.21.15.172:0/1328793765 shutdown_connections 2024-09-18T07:13:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.395+0000 7f3e26a16640 1 -- 172.21.15.172:0/1328793765 wait complete. 2024-09-18T07:13:54.408 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 80961d78-b7ae-416d-bb6f-a2ef724c6cff 3 2024-09-18T07:13:54.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.505+0000 7f6f9fecd640 1 Processor -- start 2024-09-18T07:13:54.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.505+0000 7f6f9fecd640 1 -- start start 2024-09-18T07:13:54.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9fecd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f980ac320 con 0x7f6f980aae50 2024-09-18T07:13:54.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9fecd640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f980ac590 con 0x7f6f98069050 2024-09-18T07:13:54.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9fecd640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f980aa650 con 0x7f6f98140c40 2024-09-18T07:13:54.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9dc69640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6f98069050 0x7f6f98143390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55678/0 (socket says 172.21.15.172:55678) 2024-09-18T07:13:54.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9dc69640 1 -- 172.21.15.172:0/4148498103 learned_addr learned my addr 172.21.15.172:0/4148498103 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:54.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9cc67640 1 -- 172.21.15.172:0/4148498103 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1076048391 0 0) 0x7f6f980ac590 con 0x7f6f98069050 2024-09-18T07:13:54.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9cc67640 1 -- 172.21.15.172:0/4148498103 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f84003520 con 0x7f6f98069050 2024-09-18T07:13:54.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9cc67640 1 -- 172.21.15.172:0/4148498103 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3426504594 0 0) 0x7f6f980aa650 con 0x7f6f98140c40 2024-09-18T07:13:54.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9cc67640 1 -- 172.21.15.172:0/4148498103 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f980ac590 con 0x7f6f98140c40 2024-09-18T07:13:54.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9cc67640 1 -- 172.21.15.172:0/4148498103 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 731539339 0 0) 0x7f6f980ac320 con 0x7f6f980aae50 2024-09-18T07:13:54.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9cc67640 1 -- 172.21.15.172:0/4148498103 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f980aa650 con 0x7f6f980aae50 2024-09-18T07:13:54.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9cc67640 1 -- 172.21.15.172:0/4148498103 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1878194202 0 0) 0x7f6f84003520 con 0x7f6f98069050 2024-09-18T07:13:54.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9cc67640 1 -- 172.21.15.172:0/4148498103 >> v1:172.21.15.70:6790/0 conn(0x7f6f98140c40 legacy=0x7f6f9814ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9cc67640 1 -- 172.21.15.172:0/4148498103 >> v1:172.21.15.70:6789/0 conn(0x7f6f980aae50 legacy=0x7f6f98140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9cc67640 1 -- 172.21.15.172:0/4148498103 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f980aa930 con 0x7f6f98069050 2024-09-18T07:13:54.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.506+0000 7f6f9cc67640 1 -- 172.21.15.172:0/4148498103 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6f88001690 con 0x7f6f98069050 2024-09-18T07:13:54.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.507+0000 7f6f9cc67640 1 -- 172.21.15.172:0/4148498103 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6f880018b0 con 0x7f6f98069050 2024-09-18T07:13:54.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.507+0000 7f6f9cc67640 1 -- 172.21.15.172:0/4148498103 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6f88001c80 con 0x7f6f98069050 2024-09-18T07:13:54.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.507+0000 7f6f9fecd640 1 -- 172.21.15.172:0/4148498103 >> v1:172.21.15.172:6789/0 conn(0x7f6f98069050 legacy=0x7f6f98143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.507+0000 7f6f9fecd640 1 -- 172.21.15.172:0/4148498103 shutdown_connections 2024-09-18T07:13:54.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.507+0000 7f6f9fecd640 1 -- 172.21.15.172:0/4148498103 >> 172.21.15.172:0/4148498103 conn(0x7f6f980b9370 msgr2=0x7f6f980b9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:54.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.507+0000 7f6f9fecd640 1 -- 172.21.15.172:0/4148498103 shutdown_connections 2024-09-18T07:13:54.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.507+0000 7f6f9fecd640 1 -- 172.21.15.172:0/4148498103 wait complete. 2024-09-18T07:13:54.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.508+0000 7f6f9fecd640 1 Processor -- start 2024-09-18T07:13:54.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.508+0000 7f6f9fecd640 1 -- start start 2024-09-18T07:13:54.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f9fecd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f98134af0 con 0x7f6f98069050 2024-09-18T07:13:54.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f9fecd640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f9813b570 con 0x7f6f98140c40 2024-09-18T07:13:54.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f9fecd640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f9813b850 con 0x7f6f980aae50 2024-09-18T07:13:54.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f9e46a640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6f98140c40 0x7f6f981343e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55694/0 (socket says 172.21.15.172:55694) 2024-09-18T07:13:54.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f9e46a640 1 -- 172.21.15.172:0/3694483307 learned_addr learned my addr 172.21.15.172:0/3694483307 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:54.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1473929944 0 0) 0x7f6f9813b570 con 0x7f6f98140c40 2024-09-18T07:13:54.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f7c003520 con 0x7f6f98140c40 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2729650903 0 0) 0x7f6f98134af0 con 0x7f6f98069050 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f9813b570 con 0x7f6f98069050 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3753962199 0 0) 0x7f6f9813b850 con 0x7f6f980aae50 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f98134af0 con 0x7f6f980aae50 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2608586555 0 0) 0x7f6f7c003520 con 0x7f6f98140c40 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6f9813b850 con 0x7f6f98140c40 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6f90001c00 con 0x7f6f98140c40 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2457305176 0 0) 0x7f6f9813b570 con 0x7f6f98069050 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6f7c003520 con 0x7f6f98069050 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6f88001c00 con 0x7f6f98069050 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2726215145 0 0) 0x7f6f98134af0 con 0x7f6f980aae50 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6f9813b570 con 0x7f6f980aae50 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6f94001830 con 0x7f6f980aae50 2024-09-18T07:13:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 822608105 0 0) 0x7f6f9813b850 con 0x7f6f98140c40 2024-09-18T07:13:54.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 >> v1:172.21.15.70:6790/0 conn(0x7f6f980aae50 legacy=0x7f6f98133cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 >> v1:172.21.15.70:6789/0 conn(0x7f6f98069050 legacy=0x7f6f98145170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9813a5a0 con 0x7f6f98140c40 2024-09-18T07:13:54.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.509+0000 7f6f9fecd640 1 -- 172.21.15.172:0/3694483307 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6f98145880 con 0x7f6f98140c40 2024-09-18T07:13:54.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.510+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6f90003990 con 0x7f6f98140c40 2024-09-18T07:13:54.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.510+0000 7f6f9fecd640 1 -- 172.21.15.172:0/3694483307 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6f98139ca0 con 0x7f6f98140c40 2024-09-18T07:13:54.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.510+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6f90003ce0 con 0x7f6f98140c40 2024-09-18T07:13:54.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.510+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f6f90004070 con 0x7f6f98140c40 2024-09-18T07:13:54.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.510+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (unknown 62720649 0 0) 0x7f6f90005b40 con 0x7f6f98140c40 2024-09-18T07:13:54.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.511+0000 7f6f9fecd640 1 -- 172.21.15.172:0/3694483307 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f80005080 con 0x7f6f98140c40 2024-09-18T07:13:54.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.513+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7f6f90004360 con 0x7f6f98140c40 2024-09-18T07:13:54.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.631+0000 7f6f9fecd640 1 -- 172.21.15.172:0/3694483307 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd new", "uuid": "80961d78-b7ae-416d-bb6f-a2ef724c6cff", "id": 3} v 0) -- 0x7f6f80005370 con 0x7f6f98140c40 2024-09-18T07:13:54.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.636+0000 7f6f8e7fc640 1 -- 172.21.15.172:0/3694483307 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "80961d78-b7ae-416d-bb6f-a2ef724c6cff", "id": 3}]=0 v6) ==== 112+0+2 (unknown 4137994921 0 525046306) 0x7f6f90004640 con 0x7f6f98140c40 2024-09-18T07:13:54.639 INFO:teuthology.orchestra.run.smithi172.stdout:3 2024-09-18T07:13:54.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.637+0000 7f6f9fecd640 1 -- 172.21.15.172:0/3694483307 >> v1:172.21.15.172:6789/0 conn(0x7f6f98140c40 legacy=0x7f6f981343e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.637+0000 7f6f9fecd640 1 -- 172.21.15.172:0/3694483307 shutdown_connections 2024-09-18T07:13:54.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.638+0000 7f6f9fecd640 1 -- 172.21.15.172:0/3694483307 >> 172.21.15.172:0/3694483307 conn(0x7f6f980b9370 msgr2=0x7f6f980b6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:54.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.638+0000 7f6f9fecd640 1 -- 172.21.15.172:0/3694483307 shutdown_connections 2024-09-18T07:13:54.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.638+0000 7f6f9fecd640 1 -- 172.21.15.172:0/3694483307 wait complete. 2024-09-18T07:13:54.652 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new ac72b675-4df6-4668-8461-3d0d8ac698b6 4 2024-09-18T07:13:54.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.749+0000 7fd8aad9f640 1 Processor -- start 2024-09-18T07:13:54.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.749+0000 7fd8aad9f640 1 -- start start 2024-09-18T07:13:54.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8aad9f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8a40b2020 con 0x7fd8a4068b20 2024-09-18T07:13:54.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8aad9f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8a40b2300 con 0x7fd8a40b0040 2024-09-18T07:13:54.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8aad9f640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8a40b09f0 con 0x7fd8a41408e0 2024-09-18T07:13:54.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a959c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd8a40b0040 0x7fd8a41401d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55702/0 (socket says 172.21.15.172:55702) 2024-09-18T07:13:54.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a959c640 1 -- 172.21.15.172:0/2481432800 learned_addr learned my addr 172.21.15.172:0/2481432800 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:54.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a8d9b640 1 -- 172.21.15.172:0/2481432800 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2369329124 0 0) 0x7fd8a40b2300 con 0x7fd8a40b0040 2024-09-18T07:13:54.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a8d9b640 1 -- 172.21.15.172:0/2481432800 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd890003520 con 0x7fd8a40b0040 2024-09-18T07:13:54.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a8d9b640 1 -- 172.21.15.172:0/2481432800 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1565944437 0 0) 0x7fd8a40b2020 con 0x7fd8a4068b20 2024-09-18T07:13:54.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a8d9b640 1 -- 172.21.15.172:0/2481432800 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8a40b2300 con 0x7fd8a4068b20 2024-09-18T07:13:54.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a8d9b640 1 -- 172.21.15.172:0/2481432800 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014134114 0 0) 0x7fd8a40b09f0 con 0x7fd8a41408e0 2024-09-18T07:13:54.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a8d9b640 1 -- 172.21.15.172:0/2481432800 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8a40b2020 con 0x7fd8a41408e0 2024-09-18T07:13:54.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a8d9b640 1 -- 172.21.15.172:0/2481432800 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2817474657 0 0) 0x7fd890003520 con 0x7fd8a40b0040 2024-09-18T07:13:54.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a8d9b640 1 -- 172.21.15.172:0/2481432800 >> v1:172.21.15.70:6790/0 conn(0x7fd8a41408e0 legacy=0x7fd8a414e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a8d9b640 1 -- 172.21.15.172:0/2481432800 >> v1:172.21.15.70:6789/0 conn(0x7fd8a4068b20 legacy=0x7fd8a41465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a8d9b640 1 -- 172.21.15.172:0/2481432800 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a40afb20 con 0x7fd8a40b0040 2024-09-18T07:13:54.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a8d9b640 1 -- 172.21.15.172:0/2481432800 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd894001690 con 0x7fd8a40b0040 2024-09-18T07:13:54.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a8d9b640 1 -- 172.21.15.172:0/2481432800 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd8940018b0 con 0x7fd8a40b0040 2024-09-18T07:13:54.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.750+0000 7fd8a8d9b640 1 -- 172.21.15.172:0/2481432800 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd894001c80 con 0x7fd8a40b0040 2024-09-18T07:13:54.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.751+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2481432800 >> v1:172.21.15.172:6789/0 conn(0x7fd8a40b0040 legacy=0x7fd8a41401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.751+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2481432800 shutdown_connections 2024-09-18T07:13:54.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.751+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2481432800 >> 172.21.15.172:0/2481432800 conn(0x7fd8a406d750 msgr2=0x7fd8a40b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:54.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.751+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2481432800 shutdown_connections 2024-09-18T07:13:54.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.751+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2481432800 wait complete. 2024-09-18T07:13:54.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.752+0000 7fd8aad9f640 1 Processor -- start 2024-09-18T07:13:54.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.752+0000 7fd8aad9f640 1 -- start start 2024-09-18T07:13:54.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.752+0000 7fd8aad9f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8a411f5d0 con 0x7fd8a4068b20 2024-09-18T07:13:54.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.752+0000 7fd8aad9f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8a411f8b0 con 0x7fd8a40b0040 2024-09-18T07:13:54.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.752+0000 7fd8aad9f640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8a413c860 con 0x7fd8a41408e0 2024-09-18T07:13:54.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.752+0000 7fd8a959c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd8a40b0040 0x7fd8a413b670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55714/0 (socket says 172.21.15.172:55714) 2024-09-18T07:13:54.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.752+0000 7fd8a959c640 1 -- 172.21.15.172:0/2180219066 learned_addr learned my addr 172.21.15.172:0/2180219066 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:54.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3610059025 0 0) 0x7fd8a411f8b0 con 0x7fd8a40b0040 2024-09-18T07:13:54.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd888003520 con 0x7fd8a40b0040 2024-09-18T07:13:54.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 131980269 0 0) 0x7fd8a411f5d0 con 0x7fd8a4068b20 2024-09-18T07:13:54.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8a411f8b0 con 0x7fd8a4068b20 2024-09-18T07:13:54.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 584914011 0 0) 0x7fd8a413c860 con 0x7fd8a41408e0 2024-09-18T07:13:54.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8a411f5d0 con 0x7fd8a41408e0 2024-09-18T07:13:54.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 77204208 0 0) 0x7fd888003520 con 0x7fd8a40b0040 2024-09-18T07:13:54.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd8a413c860 con 0x7fd8a40b0040 2024-09-18T07:13:54.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd894001c00 con 0x7fd8a40b0040 2024-09-18T07:13:54.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2908866151 0 0) 0x7fd8a411f8b0 con 0x7fd8a4068b20 2024-09-18T07:13:54.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd888003520 con 0x7fd8a4068b20 2024-09-18T07:13:54.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd8a0001930 con 0x7fd8a4068b20 2024-09-18T07:13:54.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1046295512 0 0) 0x7fd8a411f5d0 con 0x7fd8a41408e0 2024-09-18T07:13:54.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd8a411f8b0 con 0x7fd8a41408e0 2024-09-18T07:13:54.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 823097594 0 0) 0x7fd8a413c860 con 0x7fd8a40b0040 2024-09-18T07:13:54.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 >> v1:172.21.15.70:6790/0 conn(0x7fd8a41408e0 legacy=0x7fd8a41e4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 >> v1:172.21.15.70:6789/0 conn(0x7fd8a4068b20 legacy=0x7fd8a411d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a411deb0 con 0x7fd8a40b0040 2024-09-18T07:13:54.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2180219066 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a4073aa0 con 0x7fd8a40b0040 2024-09-18T07:13:54.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2180219066 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd8a41d94d0 con 0x7fd8a40b0040 2024-09-18T07:13:54.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd894004bf0 con 0x7fd8a40b0040 2024-09-18T07:13:54.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd894005040 con 0x7fd8a40b0040 2024-09-18T07:13:54.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7fd8940055f0 con 0x7fd8a40b0040 2024-09-18T07:13:54.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.753+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (unknown 3428741947 0 0) 0x7fd894001830 con 0x7fd8a40b0040 2024-09-18T07:13:54.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.754+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2180219066 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd88c005080 con 0x7fd8a40b0040 2024-09-18T07:13:54.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.756+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7fd8940067a0 con 0x7fd8a40b0040 2024-09-18T07:13:54.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.826+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2180219066 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd new", "uuid": "ac72b675-4df6-4668-8461-3d0d8ac698b6", "id": 4} v 0) -- 0x7fd88c005370 con 0x7fd8a40b0040 2024-09-18T07:13:54.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.830+0000 7fd89a7fc640 1 -- 172.21.15.172:0/2180219066 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac72b675-4df6-4668-8461-3d0d8ac698b6", "id": 4}]=0 v7) ==== 112+0+2 (unknown 1704895184 0 1697012583) 0x7fd894006a00 con 0x7fd8a40b0040 2024-09-18T07:13:54.832 INFO:teuthology.orchestra.run.smithi172.stdout:4 2024-09-18T07:13:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.831+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2180219066 >> v1:172.21.15.172:6789/0 conn(0x7fd8a40b0040 legacy=0x7fd8a413b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.831+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2180219066 shutdown_connections 2024-09-18T07:13:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.831+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2180219066 >> 172.21.15.172:0/2180219066 conn(0x7fd8a406d750 msgr2=0x7fd8a413e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.831+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2180219066 shutdown_connections 2024-09-18T07:13:54.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.831+0000 7fd8aad9f640 1 -- 172.21.15.172:0/2180219066 wait complete. 2024-09-18T07:13:54.844 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 5949328d-408f-4b72-8e51-db0046fbb090 5 2024-09-18T07:13:54.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.940+0000 7f48b35d3640 1 Processor -- start 2024-09-18T07:13:54.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.940+0000 7f48b35d3640 1 -- start start 2024-09-18T07:13:54.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48b35d3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48ac0ac320 con 0x7f48ac140c40 2024-09-18T07:13:54.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48b35d3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48ac0ac590 con 0x7f48ac069050 2024-09-18T07:13:54.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48b35d3640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48ac0aa650 con 0x7f48ac0aae50 2024-09-18T07:13:54.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48b136f640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f48ac069050 0x7f48ac143390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55726/0 (socket says 172.21.15.172:55726) 2024-09-18T07:13:54.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48b136f640 1 -- 172.21.15.172:0/3294859459 learned_addr learned my addr 172.21.15.172:0/3294859459 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:54.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48a3fff640 1 -- 172.21.15.172:0/3294859459 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3254234739 0 0) 0x7f48ac0ac590 con 0x7f48ac069050 2024-09-18T07:13:54.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48a3fff640 1 -- 172.21.15.172:0/3294859459 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4898003520 con 0x7f48ac069050 2024-09-18T07:13:54.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48a3fff640 1 -- 172.21.15.172:0/3294859459 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4241513044 0 0) 0x7f48ac0ac320 con 0x7f48ac140c40 2024-09-18T07:13:54.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48a3fff640 1 -- 172.21.15.172:0/3294859459 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48ac0ac590 con 0x7f48ac140c40 2024-09-18T07:13:54.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48a3fff640 1 -- 172.21.15.172:0/3294859459 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3013221297 0 0) 0x7f48ac0aa650 con 0x7f48ac0aae50 2024-09-18T07:13:54.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48a3fff640 1 -- 172.21.15.172:0/3294859459 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48ac0ac320 con 0x7f48ac0aae50 2024-09-18T07:13:54.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48a3fff640 1 -- 172.21.15.172:0/3294859459 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 462553952 0 0) 0x7f4898003520 con 0x7f48ac069050 2024-09-18T07:13:54.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48a3fff640 1 -- 172.21.15.172:0/3294859459 >> v1:172.21.15.70:6790/0 conn(0x7f48ac0aae50 legacy=0x7f48ac140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48a3fff640 1 -- 172.21.15.172:0/3294859459 >> v1:172.21.15.70:6789/0 conn(0x7f48ac140c40 legacy=0x7f48ac14ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48a3fff640 1 -- 172.21.15.172:0/3294859459 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ac0aa930 con 0x7f48ac069050 2024-09-18T07:13:54.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.941+0000 7f48a3fff640 1 -- 172.21.15.172:0/3294859459 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f489c001690 con 0x7f48ac069050 2024-09-18T07:13:54.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.942+0000 7f48a3fff640 1 -- 172.21.15.172:0/3294859459 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f489c0018b0 con 0x7f48ac069050 2024-09-18T07:13:54.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.942+0000 7f48a3fff640 1 -- 172.21.15.172:0/3294859459 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f489c001c80 con 0x7f48ac069050 2024-09-18T07:13:54.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.942+0000 7f48b35d3640 1 -- 172.21.15.172:0/3294859459 >> v1:172.21.15.172:6789/0 conn(0x7f48ac069050 legacy=0x7f48ac143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.942+0000 7f48b35d3640 1 -- 172.21.15.172:0/3294859459 shutdown_connections 2024-09-18T07:13:54.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.942+0000 7f48b35d3640 1 -- 172.21.15.172:0/3294859459 >> 172.21.15.172:0/3294859459 conn(0x7f48ac0b9370 msgr2=0x7f48ac0b9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:54.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.942+0000 7f48b35d3640 1 -- 172.21.15.172:0/3294859459 shutdown_connections 2024-09-18T07:13:54.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.942+0000 7f48b35d3640 1 -- 172.21.15.172:0/3294859459 wait complete. 2024-09-18T07:13:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.943+0000 7f48b35d3640 1 Processor -- start 2024-09-18T07:13:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.943+0000 7f48b35d3640 1 -- start start 2024-09-18T07:13:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48b35d3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48ac0b1630 con 0x7f48ac069050 2024-09-18T07:13:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48b35d3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48ac0b1910 con 0x7f48ac140c40 2024-09-18T07:13:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48b35d3640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48ac0b1bf0 con 0x7f48ac0aae50 2024-09-18T07:13:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48b136f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f48ac069050 0x7f48ac13d0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.172:45868/0 (socket says 172.21.15.172:45868) 2024-09-18T07:13:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48b1b70640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f48ac140c40 0x7f48ac1e3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55736/0 (socket says 172.21.15.172:55736) 2024-09-18T07:13:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48b136f640 1 -- 172.21.15.172:0/2228624191 learned_addr learned my addr 172.21.15.172:0/2228624191 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3969361343 0 0) 0x7f48ac0b1910 con 0x7f48ac140c40 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4890003520 con 0x7f48ac140c40 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3281704225 0 0) 0x7f48ac0b1630 con 0x7f48ac069050 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48ac0b1910 con 0x7f48ac069050 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 379021572 0 0) 0x7f48ac0b1bf0 con 0x7f48ac0aae50 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48ac0b1630 con 0x7f48ac0aae50 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 373219906 0 0) 0x7f4890003520 con 0x7f48ac140c40 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f48ac0b1bf0 con 0x7f48ac140c40 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f48a4001bd0 con 0x7f48ac140c40 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3398780676 0 0) 0x7f48ac0b1910 con 0x7f48ac069050 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4890003520 con 0x7f48ac069050 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f489c001c00 con 0x7f48ac069050 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2194751314 0 0) 0x7f48ac0b1630 con 0x7f48ac0aae50 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f48ac0b1910 con 0x7f48ac0aae50 2024-09-18T07:13:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f48a8001760 con 0x7f48ac0aae50 2024-09-18T07:13:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3462072697 0 0) 0x7f48ac0b1bf0 con 0x7f48ac140c40 2024-09-18T07:13:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 >> v1:172.21.15.70:6790/0 conn(0x7f48ac0aae50 legacy=0x7f48ac1db970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 >> v1:172.21.15.70:6789/0 conn(0x7f48ac069050 legacy=0x7f48ac13d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.944+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ac0b06b0 con 0x7f48ac140c40 2024-09-18T07:13:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.945+0000 7f48b35d3640 1 -- 172.21.15.172:0/2228624191 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f48ac0b00f0 con 0x7f48ac140c40 2024-09-18T07:13:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.945+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f48a4003970 con 0x7f48ac140c40 2024-09-18T07:13:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.945+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f48a4003d40 con 0x7f48ac140c40 2024-09-18T07:13:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.945+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f48a4004330 con 0x7f48ac140c40 2024-09-18T07:13:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.945+0000 7f48b35d3640 1 -- 172.21.15.172:0/2228624191 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f48ac0b2560 con 0x7f48ac140c40 2024-09-18T07:13:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.945+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (unknown 2989366489 0 0) 0x7f48a4004620 con 0x7f48ac140c40 2024-09-18T07:13:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.946+0000 7f48b35d3640 1 -- 172.21.15.172:0/2228624191 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ac0b04d0 con 0x7f48ac140c40 2024-09-18T07:13:54.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:54.948+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7f48ac0b04d0 con 0x7f48ac140c40 2024-09-18T07:13:55.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.018+0000 7f48b35d3640 1 -- 172.21.15.172:0/2228624191 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd new", "uuid": "5949328d-408f-4b72-8e51-db0046fbb090", "id": 5} v 0) -- 0x7f48ac06cb50 con 0x7f48ac140c40 2024-09-18T07:13:55.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.022+0000 7f48a1ffb640 1 -- 172.21.15.172:0/2228624191 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5949328d-408f-4b72-8e51-db0046fbb090", "id": 5}]=0 v8) ==== 112+0+2 (unknown 3561012735 0 1988412176) 0x7f48a4005ce0 con 0x7f48ac140c40 2024-09-18T07:13:55.025 INFO:teuthology.orchestra.run.smithi172.stdout:5 2024-09-18T07:13:55.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.023+0000 7f48b35d3640 1 -- 172.21.15.172:0/2228624191 >> v1:172.21.15.172:6789/0 conn(0x7f48ac140c40 legacy=0x7f48ac1e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.023+0000 7f48b35d3640 1 -- 172.21.15.172:0/2228624191 shutdown_connections 2024-09-18T07:13:55.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.023+0000 7f48b35d3640 1 -- 172.21.15.172:0/2228624191 >> 172.21.15.172:0/2228624191 conn(0x7f48ac0b9370 msgr2=0x7f48ac0b6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:55.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.023+0000 7f48b35d3640 1 -- 172.21.15.172:0/2228624191 shutdown_connections 2024-09-18T07:13:55.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.023+0000 7f48b35d3640 1 -- 172.21.15.172:0/2228624191 wait complete. 2024-09-18T07:13:55.036 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 84ce0e0b-9072-4ef7-9e55-f3d81283b3de 6 2024-09-18T07:13:55.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.132+0000 7faaccacd640 1 Processor -- start 2024-09-18T07:13:55.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.132+0000 7faaccacd640 1 -- start start 2024-09-18T07:13:55.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.133+0000 7faaccacd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faac80ac320 con 0x7faac80aae50 2024-09-18T07:13:55.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.133+0000 7faaccacd640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faac80ac590 con 0x7faac8140c40 2024-09-18T07:13:55.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.133+0000 7faaccacd640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faac80aa650 con 0x7faac8069050 2024-09-18T07:13:55.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.133+0000 7faac6d9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7faac8140c40 0x7faac814ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55748/0 (socket says 172.21.15.172:55748) 2024-09-18T07:13:55.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.133+0000 7faac6d9d640 1 -- 172.21.15.172:0/3750848047 learned_addr learned my addr 172.21.15.172:0/3750848047 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:55.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.133+0000 7faac559a640 1 -- 172.21.15.172:0/3750848047 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3766125363 0 0) 0x7faac80ac590 con 0x7faac8140c40 2024-09-18T07:13:55.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.134+0000 7faac559a640 1 -- 172.21.15.172:0/3750848047 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faab4003520 con 0x7faac8140c40 2024-09-18T07:13:55.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.134+0000 7faac559a640 1 -- 172.21.15.172:0/3750848047 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2724874006 0 0) 0x7faac80aa650 con 0x7faac8069050 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.134+0000 7faac559a640 1 -- 172.21.15.172:0/3750848047 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faac80ac590 con 0x7faac8069050 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.134+0000 7faac559a640 1 -- 172.21.15.172:0/3750848047 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1283199634 0 0) 0x7faac80ac320 con 0x7faac80aae50 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.134+0000 7faac559a640 1 -- 172.21.15.172:0/3750848047 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faac80aa650 con 0x7faac80aae50 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.134+0000 7faac559a640 1 -- 172.21.15.172:0/3750848047 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3565415629 0 0) 0x7faab4003520 con 0x7faac8140c40 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.134+0000 7faac559a640 1 -- 172.21.15.172:0/3750848047 >> v1:172.21.15.70:6790/0 conn(0x7faac8069050 legacy=0x7faac8143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.134+0000 7faac559a640 1 -- 172.21.15.172:0/3750848047 >> v1:172.21.15.70:6789/0 conn(0x7faac80aae50 legacy=0x7faac8140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.134+0000 7faac559a640 1 -- 172.21.15.172:0/3750848047 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac80aa930 con 0x7faac8140c40 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.134+0000 7faac559a640 1 -- 172.21.15.172:0/3750848047 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7faac0001a10 con 0x7faac8140c40 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.134+0000 7faac559a640 1 -- 172.21.15.172:0/3750848047 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7faac0001c30 con 0x7faac8140c40 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.134+0000 7faac559a640 1 -- 172.21.15.172:0/3750848047 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7faac0002000 con 0x7faac8140c40 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.134+0000 7faaccacd640 1 -- 172.21.15.172:0/3750848047 >> v1:172.21.15.172:6789/0 conn(0x7faac8140c40 legacy=0x7faac814ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.135+0000 7faaccacd640 1 -- 172.21.15.172:0/3750848047 shutdown_connections 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.135+0000 7faaccacd640 1 -- 172.21.15.172:0/3750848047 >> 172.21.15.172:0/3750848047 conn(0x7faac80b9370 msgr2=0x7faac80b9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:55.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.135+0000 7faaccacd640 1 -- 172.21.15.172:0/3750848047 shutdown_connections 2024-09-18T07:13:55.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.135+0000 7faaccacd640 1 -- 172.21.15.172:0/3750848047 wait complete. 2024-09-18T07:13:55.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.136+0000 7faaccacd640 1 Processor -- start 2024-09-18T07:13:55.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.136+0000 7faaccacd640 1 -- start start 2024-09-18T07:13:55.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.136+0000 7faaccacd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faac8135a90 con 0x7faac8069050 2024-09-18T07:13:55.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.136+0000 7faaccacd640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faac813a7c0 con 0x7faac8140c40 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.136+0000 7faaccacd640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faac813aaa0 con 0x7faac80aae50 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.136+0000 7faac6d9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7faac8140c40 0x7faac8135380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55756/0 (socket says 172.21.15.172:55756) 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.136+0000 7faac6d9d640 1 -- 172.21.15.172:0/2627767506 learned_addr learned my addr 172.21.15.172:0/2627767506 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2797707133 0 0) 0x7faac813a7c0 con 0x7faac8140c40 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faaa8003520 con 0x7faac8140c40 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3040160996 0 0) 0x7faac8135a90 con 0x7faac8069050 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faac813a7c0 con 0x7faac8069050 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2536179695 0 0) 0x7faac813aaa0 con 0x7faac80aae50 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faac8135a90 con 0x7faac80aae50 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3985458551 0 0) 0x7faaa8003520 con 0x7faac8140c40 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faac813aaa0 con 0x7faac8140c40 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7faac0001f80 con 0x7faac8140c40 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2638753978 0 0) 0x7faac8135a90 con 0x7faac80aae50 2024-09-18T07:13:55.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faaa8003520 con 0x7faac80aae50 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7faab0001830 con 0x7faac80aae50 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1830523915 0 0) 0x7faac813a7c0 con 0x7faac8069050 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faac8135a90 con 0x7faac8069050 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 593103066 0 0) 0x7faac813aaa0 con 0x7faac8140c40 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 >> v1:172.21.15.70:6790/0 conn(0x7faac80aae50 legacy=0x7faac8134c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 >> v1:172.21.15.70:6789/0 conn(0x7faac8069050 legacy=0x7faac81d55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac813ad80 con 0x7faac8140c40 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faaccacd640 1 -- 172.21.15.172:0/2627767506 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7faac8139510 con 0x7faac8140c40 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faaccacd640 1 -- 172.21.15.172:0/2627767506 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7faac8147360 con 0x7faac8140c40 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7faac0004e10 con 0x7faac8140c40 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7faac0005160 con 0x7faac8140c40 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7faac0005690 con 0x7faac8140c40 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.137+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (unknown 3392293994 0 0) 0x7faac0001bd0 con 0x7faac8140c40 2024-09-18T07:13:55.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.138+0000 7faaccacd640 1 -- 172.21.15.172:0/2627767506 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaac005080 con 0x7faac8140c40 2024-09-18T07:13:55.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.140+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7faac0005980 con 0x7faac8140c40 2024-09-18T07:13:55.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.211+0000 7faaccacd640 1 -- 172.21.15.172:0/2627767506 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd new", "uuid": "84ce0e0b-9072-4ef7-9e55-f3d81283b3de", "id": 6} v 0) -- 0x7faaac005370 con 0x7faac8140c40 2024-09-18T07:13:55.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.214+0000 7faabeffd640 1 -- 172.21.15.172:0/2627767506 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "84ce0e0b-9072-4ef7-9e55-f3d81283b3de", "id": 6}]=0 v9) ==== 112+0+2 (unknown 1406417856 0 1113811849) 0x7faac0005b60 con 0x7faac8140c40 2024-09-18T07:13:55.217 INFO:teuthology.orchestra.run.smithi172.stdout:6 2024-09-18T07:13:55.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.215+0000 7faaccacd640 1 -- 172.21.15.172:0/2627767506 >> v1:172.21.15.172:6789/0 conn(0x7faac8140c40 legacy=0x7faac8135380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.215+0000 7faaccacd640 1 -- 172.21.15.172:0/2627767506 shutdown_connections 2024-09-18T07:13:55.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.215+0000 7faaccacd640 1 -- 172.21.15.172:0/2627767506 >> 172.21.15.172:0/2627767506 conn(0x7faac80b9370 msgr2=0x7faac80b6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:55.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.215+0000 7faaccacd640 1 -- 172.21.15.172:0/2627767506 shutdown_connections 2024-09-18T07:13:55.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.215+0000 7faaccacd640 1 -- 172.21.15.172:0/2627767506 wait complete. 2024-09-18T07:13:55.228 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 96889def-b533-4738-bb6d-dd8eb99e1e45 7 2024-09-18T07:13:55.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.325+0000 7fb9d654b640 1 Processor -- start 2024-09-18T07:13:55.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.325+0000 7fb9d654b640 1 -- start start 2024-09-18T07:13:55.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.326+0000 7fb9d654b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d00ac320 con 0x7fb9d0140c40 2024-09-18T07:13:55.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.326+0000 7fb9d654b640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d00ac590 con 0x7fb9d0069050 2024-09-18T07:13:55.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.326+0000 7fb9d654b640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d00aa650 con 0x7fb9d00aae50 2024-09-18T07:13:55.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.326+0000 7fb9cffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb9d0069050 0x7fb9d0143390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55764/0 (socket says 172.21.15.172:55764) 2024-09-18T07:13:55.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.326+0000 7fb9cffff640 1 -- 172.21.15.172:0/1416000965 learned_addr learned my addr 172.21.15.172:0/1416000965 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:55.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.326+0000 7fb9ceffd640 1 -- 172.21.15.172:0/1416000965 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3042362130 0 0) 0x7fb9d00ac590 con 0x7fb9d0069050 2024-09-18T07:13:55.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.326+0000 7fb9ceffd640 1 -- 172.21.15.172:0/1416000965 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9bc003520 con 0x7fb9d0069050 2024-09-18T07:13:55.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9ceffd640 1 -- 172.21.15.172:0/1416000965 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3186472886 0 0) 0x7fb9d00ac320 con 0x7fb9d0140c40 2024-09-18T07:13:55.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9ceffd640 1 -- 172.21.15.172:0/1416000965 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9d00ac590 con 0x7fb9d0140c40 2024-09-18T07:13:55.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9ceffd640 1 -- 172.21.15.172:0/1416000965 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1294213137 0 0) 0x7fb9bc003520 con 0x7fb9d0069050 2024-09-18T07:13:55.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9ceffd640 1 -- 172.21.15.172:0/1416000965 >> v1:172.21.15.70:6790/0 conn(0x7fb9d00aae50 legacy=0x7fb9d0140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9ceffd640 1 -- 172.21.15.172:0/1416000965 >> v1:172.21.15.70:6789/0 conn(0x7fb9d0140c40 legacy=0x7fb9d014ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9ceffd640 1 -- 172.21.15.172:0/1416000965 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d00aa930 con 0x7fb9d0069050 2024-09-18T07:13:55.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9ceffd640 1 -- 172.21.15.172:0/1416000965 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb9c0001690 con 0x7fb9d0069050 2024-09-18T07:13:55.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9ceffd640 1 -- 172.21.15.172:0/1416000965 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb9c0001830 con 0x7fb9d0069050 2024-09-18T07:13:55.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9ceffd640 1 -- 172.21.15.172:0/1416000965 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb9c0001c40 con 0x7fb9d0069050 2024-09-18T07:13:55.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9d654b640 1 -- 172.21.15.172:0/1416000965 >> v1:172.21.15.172:6789/0 conn(0x7fb9d0069050 legacy=0x7fb9d0143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9d654b640 1 -- 172.21.15.172:0/1416000965 shutdown_connections 2024-09-18T07:13:55.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9d654b640 1 -- 172.21.15.172:0/1416000965 >> 172.21.15.172:0/1416000965 conn(0x7fb9d00b9370 msgr2=0x7fb9d00b9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:55.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9d654b640 1 -- 172.21.15.172:0/1416000965 shutdown_connections 2024-09-18T07:13:55.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.327+0000 7fb9d654b640 1 -- 172.21.15.172:0/1416000965 wait complete. 2024-09-18T07:13:55.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.328+0000 7fb9d654b640 1 Processor -- start 2024-09-18T07:13:55.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.328+0000 7fb9d654b640 1 -- start start 2024-09-18T07:13:55.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9d654b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d00acb60 con 0x7fb9d0069050 2024-09-18T07:13:55.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9d654b640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d00b2880 con 0x7fb9d00aae50 2024-09-18T07:13:55.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9d654b640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d01d8bc0 con 0x7fb9d0140c40 2024-09-18T07:13:55.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9cf7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb9d00aae50 0x7fb9d013bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55772/0 (socket says 172.21.15.172:55772) 2024-09-18T07:13:55.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9cf7fe640 1 -- 172.21.15.172:0/1833244597 learned_addr learned my addr 172.21.15.172:0/1833244597 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-09-18T07:13:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 962734618 0 0) 0x7fb9d00b2880 con 0x7fb9d00aae50 2024-09-18T07:13:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9b0003520 con 0x7fb9d00aae50 2024-09-18T07:13:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1058326605 0 0) 0x7fb9d00acb60 con 0x7fb9d0069050 2024-09-18T07:13:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9d00b2880 con 0x7fb9d0069050 2024-09-18T07:13:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2771895851 0 0) 0x7fb9d01d8bc0 con 0x7fb9d0140c40 2024-09-18T07:13:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9d00acb60 con 0x7fb9d0140c40 2024-09-18T07:13:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1230163170 0 0) 0x7fb9b0003520 con 0x7fb9d00aae50 2024-09-18T07:13:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb9d01d8bc0 con 0x7fb9d00aae50 2024-09-18T07:13:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.329+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb9b8001940 con 0x7fb9d00aae50 2024-09-18T07:13:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3831522582 0 0) 0x7fb9d00b2880 con 0x7fb9d0069050 2024-09-18T07:13:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb9b0003520 con 0x7fb9d0069050 2024-09-18T07:13:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb9c0001b80 con 0x7fb9d0069050 2024-09-18T07:13:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 586644773 0 0) 0x7fb9d00acb60 con 0x7fb9d0140c40 2024-09-18T07:13:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb9d00b2880 con 0x7fb9d0140c40 2024-09-18T07:13:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb9c8001bd0 con 0x7fb9d0140c40 2024-09-18T07:13:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3789080673 0 0) 0x7fb9d01d8bc0 con 0x7fb9d00aae50 2024-09-18T07:13:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 >> v1:172.21.15.70:6790/0 conn(0x7fb9d0140c40 legacy=0x7fb9d013c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 >> v1:172.21.15.70:6789/0 conn(0x7fb9d0069050 legacy=0x7fb9d01d84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d01d8d90 con 0x7fb9d00aae50 2024-09-18T07:13:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9d654b640 1 -- 172.21.15.172:0/1833244597 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d01e3d50 con 0x7fb9d00aae50 2024-09-18T07:13:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9d654b640 1 -- 172.21.15.172:0/1833244597 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb9d01e4100 con 0x7fb9d00aae50 2024-09-18T07:13:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb9b8001c60 con 0x7fb9d00aae50 2024-09-18T07:13:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb9b8003a60 con 0x7fb9d00aae50 2024-09-18T07:13:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7fb9b8004050 con 0x7fb9d00aae50 2024-09-18T07:13:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.330+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (unknown 3362520299 0 0) 0x7fb9b8006890 con 0x7fb9d00aae50 2024-09-18T07:13:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.331+0000 7fb9d654b640 1 -- 172.21.15.172:0/1833244597 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b4005080 con 0x7fb9d00aae50 2024-09-18T07:13:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.333+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7fb9b8006bb0 con 0x7fb9d00aae50 2024-09-18T07:13:55.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.402+0000 7fb9d654b640 1 -- 172.21.15.172:0/1833244597 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd new", "uuid": "96889def-b533-4738-bb6d-dd8eb99e1e45", "id": 7} v 0) -- 0x7fb9b4005370 con 0x7fb9d00aae50 2024-09-18T07:13:55.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.404+0000 7fb9ccff9640 1 -- 172.21.15.172:0/1833244597 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "96889def-b533-4738-bb6d-dd8eb99e1e45", "id": 7}]=0 v10) ==== 112+0+2 (unknown 1816733979 0 1371665406) 0x7fb9b8006230 con 0x7fb9d00aae50 2024-09-18T07:13:55.407 INFO:teuthology.orchestra.run.smithi172.stdout:7 2024-09-18T07:13:55.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.405+0000 7fb9d654b640 1 -- 172.21.15.172:0/1833244597 >> v1:172.21.15.172:6789/0 conn(0x7fb9d00aae50 legacy=0x7fb9d013bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.405+0000 7fb9d654b640 1 -- 172.21.15.172:0/1833244597 shutdown_connections 2024-09-18T07:13:55.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.405+0000 7fb9d654b640 1 -- 172.21.15.172:0/1833244597 >> 172.21.15.172:0/1833244597 conn(0x7fb9d00b9370 msgr2=0x7fb9d00b6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:55.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.406+0000 7fb9d654b640 1 -- 172.21.15.172:0/1833244597 shutdown_connections 2024-09-18T07:13:55.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-18T07:13:55.406+0000 7fb9d654b640 1 -- 172.21.15.172:0/1833244597 wait complete. 2024-09-18T07:13:55.418 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-18T07:13:55.418 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-18T07:13:55.421 INFO:tasks.ceph.osd.0:Started 2024-09-18T07:13:55.421 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-18T07:13:55.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-18T07:13:55.423 INFO:tasks.ceph.osd.1:Started 2024-09-18T07:13:55.423 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-18T07:13:55.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-18T07:13:55.425 INFO:tasks.ceph.osd.2:Started 2024-09-18T07:13:55.425 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-18T07:13:55.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-18T07:13:55.428 INFO:tasks.ceph.osd.3:Started 2024-09-18T07:13:55.428 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-18T07:13:55.428 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-18T07:13:55.430 INFO:tasks.ceph.osd.4:Started 2024-09-18T07:13:55.430 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-18T07:13:55.430 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-18T07:13:55.432 INFO:tasks.ceph.osd.5:Started 2024-09-18T07:13:55.432 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-18T07:13:55.433 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-18T07:13:55.435 INFO:tasks.ceph.osd.6:Started 2024-09-18T07:13:55.435 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-18T07:13:55.435 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-18T07:13:55.437 INFO:tasks.ceph.osd.7:Started 2024-09-18T07:13:55.438 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T07:13:55.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.552+0000 7fadb579d640 1 Processor -- start 2024-09-18T07:13:55.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.552+0000 7fadb579d640 1 -- start start 2024-09-18T07:13:55.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.553+0000 7fadb579d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb00736a0 con 0x7fadb013c770 2024-09-18T07:13:55.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.553+0000 7fadb579d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb0072010 con 0x7fadb00b0e10 2024-09-18T07:13:55.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.553+0000 7fadaf7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fadb013c770 0x7fadb013f4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41260/0 (socket says 172.21.15.70:41260) 2024-09-18T07:13:55.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.553+0000 7fadb579d640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb00af6a0 con 0x7fadb013fbe0 2024-09-18T07:13:55.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.553+0000 7fadb4f9c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fadb013fbe0 0x7fadb014ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:50388/0 (socket says 172.21.15.70:50388) 2024-09-18T07:13:55.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.553+0000 7fadb4f9c640 1 -- 172.21.15.70:0/1578897793 learned_addr learned my addr 172.21.15.70:0/1578897793 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:13:55.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.553+0000 7fadaeffd640 1 -- 172.21.15.70:0/1578897793 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2556954953 0 0) 0x7fadb00af6a0 con 0x7fadb013fbe0 2024-09-18T07:13:55.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.553+0000 7fadaeffd640 1 -- 172.21.15.70:0/1578897793 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad98004630 con 0x7fadb013fbe0 2024-09-18T07:13:55.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.553+0000 7fadaeffd640 1 -- 172.21.15.70:0/1578897793 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2014240577 0 0) 0x7fadb00736a0 con 0x7fadb013c770 2024-09-18T07:13:55.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.553+0000 7fadaeffd640 1 -- 172.21.15.70:0/1578897793 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadb00af6a0 con 0x7fadb013c770 2024-09-18T07:13:55.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.553+0000 7fadaeffd640 1 -- 172.21.15.70:0/1578897793 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810647826 0 0) 0x7fadb0072010 con 0x7fadb00b0e10 2024-09-18T07:13:55.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.553+0000 7fadaeffd640 1 -- 172.21.15.70:0/1578897793 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadb00736a0 con 0x7fadb00b0e10 2024-09-18T07:13:55.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.553+0000 7fadaeffd640 1 -- 172.21.15.70:0/1578897793 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2821415674 0 0) 0x7fadb00736a0 con 0x7fadb00b0e10 2024-09-18T07:13:55.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.554+0000 7fadaeffd640 1 -- 172.21.15.70:0/1578897793 >> v1:172.21.15.70:6790/0 conn(0x7fadb013fbe0 legacy=0x7fadb014ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.554+0000 7fadaeffd640 1 -- 172.21.15.70:0/1578897793 >> v1:172.21.15.70:6789/0 conn(0x7fadb013c770 legacy=0x7fadb013f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.554+0000 7fadaeffd640 1 -- 172.21.15.70:0/1578897793 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb0072530 con 0x7fadb00b0e10 2024-09-18T07:13:55.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.554+0000 7fadaeffd640 1 -- 172.21.15.70:0/1578897793 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fada00015b0 con 0x7fadb00b0e10 2024-09-18T07:13:55.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.554+0000 7fadaeffd640 1 -- 172.21.15.70:0/1578897793 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fada0001750 con 0x7fadb00b0e10 2024-09-18T07:13:55.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.554+0000 7fadaeffd640 1 -- 172.21.15.70:0/1578897793 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fada0001be0 con 0x7fadb00b0e10 2024-09-18T07:13:55.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.554+0000 7fadb579d640 1 -- 172.21.15.70:0/1578897793 >> v1:172.21.15.172:6789/0 conn(0x7fadb00b0e10 legacy=0x7fadb0071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.554+0000 7fadb579d640 1 -- 172.21.15.70:0/1578897793 shutdown_connections 2024-09-18T07:13:55.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.554+0000 7fadb579d640 1 -- 172.21.15.70:0/1578897793 >> 172.21.15.70:0/1578897793 conn(0x7fadb00b1c90 msgr2=0x7fadb00b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:55.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.554+0000 7fadb579d640 1 -- 172.21.15.70:0/1578897793 shutdown_connections 2024-09-18T07:13:55.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.554+0000 7fadb579d640 1 -- 172.21.15.70:0/1578897793 wait complete. 2024-09-18T07:13:55.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.555+0000 7fadb579d640 1 Processor -- start 2024-09-18T07:13:55.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.555+0000 7fadb579d640 1 -- start start 2024-09-18T07:13:55.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.556+0000 7fadb579d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb00aa4e0 con 0x7fadb00b0e10 2024-09-18T07:13:55.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.556+0000 7fadb579d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb00aa7c0 con 0x7fadb013c770 2024-09-18T07:13:55.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.556+0000 7fadb579d640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb01418c0 con 0x7fadb013fbe0 2024-09-18T07:13:55.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.556+0000 7fadaffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fadb00b0e10 0x7fadb0142480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41264/0 (socket says 172.21.15.70:41264) 2024-09-18T07:13:55.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.556+0000 7fadaffff640 1 -- 172.21.15.70:0/127045027 learned_addr learned my addr 172.21.15.70:0/127045027 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:13:55.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.556+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2524984237 0 0) 0x7fadb00aa4e0 con 0x7fadb00b0e10 2024-09-18T07:13:55.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.556+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad9c003520 con 0x7fadb00b0e10 2024-09-18T07:13:55.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.556+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2334574841 0 0) 0x7fadb01418c0 con 0x7fadb013fbe0 2024-09-18T07:13:55.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.556+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadb00aa4e0 con 0x7fadb013fbe0 2024-09-18T07:13:55.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.556+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3344812219 0 0) 0x7fadb00aa7c0 con 0x7fadb013c770 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.556+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadb01418c0 con 0x7fadb013c770 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3481315495 0 0) 0x7fad9c003520 con 0x7fadb00b0e10 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fadb00aa7c0 con 0x7fadb00b0e10 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fada00035f0 con 0x7fadb00b0e10 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2847020103 0 0) 0x7fadb00aa4e0 con 0x7fadb013fbe0 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fad9c003520 con 0x7fadb013fbe0 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fada8001a70 con 0x7fadb013fbe0 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2502833028 0 0) 0x7fadb01418c0 con 0x7fadb013c770 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fadb00aa4e0 con 0x7fadb013c770 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fada4001230 con 0x7fadb013c770 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3226272702 0 0) 0x7fadb00aa7c0 con 0x7fadb00b0e10 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 >> v1:172.21.15.70:6790/0 conn(0x7fadb013fbe0 legacy=0x7fadb01d7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 >> v1:172.21.15.172:6789/0 conn(0x7fadb013c770 legacy=0x7fadb0138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb0143780 con 0x7fadb00b0e10 2024-09-18T07:13:55.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadb579d640 1 -- 172.21.15.70:0/127045027 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fadb0138a10 con 0x7fadb00b0e10 2024-09-18T07:13:55.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadb579d640 1 -- 172.21.15.70:0/127045027 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fadb01d84c0 con 0x7fadb00b0e10 2024-09-18T07:13:55.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fada0001690 con 0x7fadb00b0e10 2024-09-18T07:13:55.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fada0003c60 con 0x7fadb00b0e10 2024-09-18T07:13:55.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7fada0004250 con 0x7fadb00b0e10 2024-09-18T07:13:55.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.557+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (unknown 3868054303 0 0) 0x7fada0006b50 con 0x7fadb00b0e10 2024-09-18T07:13:55.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.558+0000 7fadb579d640 1 -- 172.21.15.70:0/127045027 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad90005080 con 0x7fadb00b0e10 2024-09-18T07:13:55.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.560+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7fada0003c60 con 0x7fadb00b0e10 2024-09-18T07:13:55.623 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.620+0000 7fadb579d640 1 -- 172.21.15.70:0/127045027 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad90005370 con 0x7fadb00b0e10 2024-09-18T07:13:55.623 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.621+0000 7fadacff9640 1 -- 172.21.15.70:0/127045027 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (unknown 4040575075 0 3700489896) 0x7fada0006610 con 0x7fadb00b0e10 2024-09-18T07:13:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:13:55.623 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":10,"fsid":"1fa528ad-1630-4e95-9001-978a53f2a42c","created":"2024-09-18T07:13:53.234326+0000","modified":"2024-09-18T07:13:55.403239+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T07:13:55.403239+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a4a2c5fa-37b8-4776-bf81-f142248da0da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3fab2225-9881-4caf-8142-41b857b9b99a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5d6764cd-e7a4-45df-9cfb-88d9e9a2675f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"80961d78-b7ae-416d-bb6f-a2ef724c6cff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ac72b675-4df6-4668-8461-3d0d8ac698b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5949328d-408f-4b72-8e51-db0046fbb090","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"84ce0e0b-9072-4ef7-9e55-f3d81283b3de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"96889def-b533-4738-bb6d-dd8eb99e1e45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T07:13:55.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.622+0000 7fadb579d640 1 -- 172.21.15.70:0/127045027 >> v1:172.21.15.70:6789/0 conn(0x7fadb00b0e10 legacy=0x7fadb0142480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:55.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.622+0000 7fadb579d640 1 -- 172.21.15.70:0/127045027 shutdown_connections 2024-09-18T07:13:55.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.622+0000 7fadb579d640 1 -- 172.21.15.70:0/127045027 >> 172.21.15.70:0/127045027 conn(0x7fadb00b1c90 msgr2=0x7fadb00b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:55.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.622+0000 7fadb579d640 1 -- 172.21.15.70:0/127045027 shutdown_connections 2024-09-18T07:13:55.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:13:55.622+0000 7fadb579d640 1 -- 172.21.15.70:0/127045027 wait complete. 2024-09-18T07:13:55.634 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-18T07:13:55.635 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-18T07:13:56.440 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T07:13:56.441 INFO:tasks.ceph.osd.7.smithi172.stderr:2024-09-18T07:13:56.438+0000 7fa658c0f640 -1 Falling back to public interface 2024-09-18T07:13:56.530 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.527+0000 7f546b510640 1 Processor -- start 2024-09-18T07:13:56.530 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.527+0000 7f546b510640 1 -- start start 2024-09-18T07:13:56.532 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.528+0000 7f546b510640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f546406f610 con 0x7f54640b2cc0 2024-09-18T07:13:56.532 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.528+0000 7f546b510640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54640af6c0 con 0x7f5464147190 2024-09-18T07:13:56.532 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.528+0000 7f546b510640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f546406ee10 con 0x7f54640b3690 2024-09-18T07:13:56.533 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.528+0000 7f54692ac640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f54640b2cc0 0x7f5464140740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41276/0 (socket says 172.21.15.70:41276) 2024-09-18T07:13:56.533 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.528+0000 7f54692ac640 1 -- 172.21.15.70:0/143873437 learned_addr learned my addr 172.21.15.70:0/143873437 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:13:56.533 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.529+0000 7f5457fff640 1 -- 172.21.15.70:0/143873437 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 993212098 0 0) 0x7f546406f610 con 0x7f54640b2cc0 2024-09-18T07:13:56.533 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.529+0000 7f5457fff640 1 -- 172.21.15.70:0/143873437 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5450003520 con 0x7f54640b2cc0 2024-09-18T07:13:56.533 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.529+0000 7f5457fff640 1 -- 172.21.15.70:0/143873437 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2328868916 0 0) 0x7f546406ee10 con 0x7f54640b3690 2024-09-18T07:13:56.533 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.529+0000 7f5457fff640 1 -- 172.21.15.70:0/143873437 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f546406f610 con 0x7f54640b3690 2024-09-18T07:13:56.533 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.529+0000 7f5457fff640 1 -- 172.21.15.70:0/143873437 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1265863645 0 0) 0x7f54640af6c0 con 0x7f5464147190 2024-09-18T07:13:56.533 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.529+0000 7f5457fff640 1 -- 172.21.15.70:0/143873437 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f546406ee10 con 0x7f5464147190 2024-09-18T07:13:56.533 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.529+0000 7f5457fff640 1 -- 172.21.15.70:0/143873437 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 688245335 0 0) 0x7f546406f610 con 0x7f54640b3690 2024-09-18T07:13:56.533 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.529+0000 7f5457fff640 1 -- 172.21.15.70:0/143873437 >> v1:172.21.15.172:6789/0 conn(0x7f5464147190 legacy=0x7f546414ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:56.533 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.529+0000 7f5457fff640 1 -- 172.21.15.70:0/143873437 >> v1:172.21.15.70:6789/0 conn(0x7f54640b2cc0 legacy=0x7f5464140740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:56.533 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.529+0000 7f5457fff640 1 -- 172.21.15.70:0/143873437 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5464140e50 con 0x7f54640b3690 2024-09-18T07:13:56.533 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.529+0000 7f5457fff640 1 -- 172.21.15.70:0/143873437 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f54580015a0 con 0x7f54640b3690 2024-09-18T07:13:56.534 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.529+0000 7f5457fff640 1 -- 172.21.15.70:0/143873437 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5458001740 con 0x7f54640b3690 2024-09-18T07:13:56.534 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.529+0000 7f5457fff640 1 -- 172.21.15.70:0/143873437 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f5458001bd0 con 0x7f54640b3690 2024-09-18T07:13:56.534 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.530+0000 7f546b510640 1 -- 172.21.15.70:0/143873437 >> v1:172.21.15.70:6790/0 conn(0x7f54640b3690 legacy=0x7f5464146a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:56.534 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.530+0000 7f546b510640 1 -- 172.21.15.70:0/143873437 shutdown_connections 2024-09-18T07:13:56.534 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.530+0000 7f546b510640 1 -- 172.21.15.70:0/143873437 >> 172.21.15.70:0/143873437 conn(0x7f54640b3b40 msgr2=0x7f54640b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:56.534 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.530+0000 7f546b510640 1 -- 172.21.15.70:0/143873437 shutdown_connections 2024-09-18T07:13:56.534 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.530+0000 7f546b510640 1 -- 172.21.15.70:0/143873437 wait complete. 2024-09-18T07:13:56.537 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.531+0000 7f546b510640 1 Processor -- start 2024-09-18T07:13:56.537 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.531+0000 7f546b510640 1 -- start start 2024-09-18T07:13:56.537 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.531+0000 7f546b510640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5464137d30 con 0x7f54640b3690 2024-09-18T07:13:56.537 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.531+0000 7f546b510640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5464138010 con 0x7f54640b2cc0 2024-09-18T07:13:56.537 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.531+0000 7f546b510640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54641382f0 con 0x7f5464147190 2024-09-18T07:13:56.537 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.531+0000 7f5469aad640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f5464147190 0x7f54641dc260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:50424/0 (socket says 172.21.15.70:50424) 2024-09-18T07:13:56.537 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.531+0000 7f5469aad640 1 -- 172.21.15.70:0/233522633 learned_addr learned my addr 172.21.15.70:0/233522633 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:13:56.537 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.531+0000 7f5468aab640 1 --1- 172.21.15.70:0/233522633 >> v1:172.21.15.70:6789/0 conn(0x7f54640b3690 0x7f54641dbb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41288/0 (socket says 172.21.15.70:41288) 2024-09-18T07:13:56.537 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2766734262 0 0) 0x7f5464137d30 con 0x7f54640b3690 2024-09-18T07:13:56.537 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5448003520 con 0x7f54640b3690 2024-09-18T07:13:56.537 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2195655067 0 0) 0x7f54641382f0 con 0x7f5464147190 2024-09-18T07:13:56.537 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5464137d30 con 0x7f5464147190 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1796435083 0 0) 0x7f5464138010 con 0x7f54640b2cc0 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54641382f0 con 0x7f54640b2cc0 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 249503081 0 0) 0x7f5448003520 con 0x7f54640b3690 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5464138010 con 0x7f54640b3690 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f5458001bd0 con 0x7f54640b3690 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3517482156 0 0) 0x7f5464137d30 con 0x7f5464147190 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5448003520 con 0x7f5464147190 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f545c001bd0 con 0x7f5464147190 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1610110931 0 0) 0x7f54641382f0 con 0x7f54640b2cc0 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5464137d30 con 0x7f54640b2cc0 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f5460001230 con 0x7f54640b2cc0 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3805124243 0 0) 0x7f5464138010 con 0x7f54640b3690 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 >> v1:172.21.15.70:6790/0 conn(0x7f5464147190 legacy=0x7f54641dc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:56.538 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 >> v1:172.21.15.172:6789/0 conn(0x7f54640b2cc0 legacy=0x7f5464136250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:56.539 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54640748e0 con 0x7f54640b3690 2024-09-18T07:13:56.539 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f546b510640 1 -- 172.21.15.70:0/233522633 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5464136960 con 0x7f54640b3690 2024-09-18T07:13:56.539 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f54580038a0 con 0x7f54640b3690 2024-09-18T07:13:56.539 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f5458003c30 con 0x7f54640b3690 2024-09-18T07:13:56.539 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f54580041e0 con 0x7f54640b3690 2024-09-18T07:13:56.539 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f546b510640 1 -- 172.21.15.70:0/233522633 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f5464143c80 con 0x7f54640b3690 2024-09-18T07:13:56.539 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.532+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (unknown 3868054303 0 0) 0x7f5458005e50 con 0x7f54640b3690 2024-09-18T07:13:56.539 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.533+0000 7f546b510640 1 -- 172.21.15.70:0/233522633 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54640b14f0 con 0x7f54640b3690 2024-09-18T07:13:56.539 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.535+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (unknown 1092875540 0 1702466736) 0x7f5458005780 con 0x7f54640b3690 2024-09-18T07:13:56.597 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.594+0000 7f546b510640 1 -- 172.21.15.70:0/233522633 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f546413e5b0 con 0x7f54640b3690 2024-09-18T07:13:56.597 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.594+0000 7f5455ffb640 1 -- 172.21.15.70:0/233522633 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (unknown 4040575075 0 3700489896) 0x7f5458005960 con 0x7f54640b3690 2024-09-18T07:13:56.597 INFO:teuthology.misc.health.smithi070.stdout: 2024-09-18T07:13:56.597 INFO:teuthology.misc.health.smithi070.stdout:{"epoch":10,"fsid":"1fa528ad-1630-4e95-9001-978a53f2a42c","created":"2024-09-18T07:13:53.234326+0000","modified":"2024-09-18T07:13:55.403239+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T07:13:55.403239+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a4a2c5fa-37b8-4776-bf81-f142248da0da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3fab2225-9881-4caf-8142-41b857b9b99a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5d6764cd-e7a4-45df-9cfb-88d9e9a2675f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"80961d78-b7ae-416d-bb6f-a2ef724c6cff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ac72b675-4df6-4668-8461-3d0d8ac698b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5949328d-408f-4b72-8e51-db0046fbb090","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"84ce0e0b-9072-4ef7-9e55-f3d81283b3de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"96889def-b533-4738-bb6d-dd8eb99e1e45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T07:13:56.598 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.595+0000 7f546b510640 1 -- 172.21.15.70:0/233522633 >> v1:172.21.15.70:6789/0 conn(0x7f54640b3690 legacy=0x7f54641dbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:13:56.598 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.595+0000 7f546b510640 1 -- 172.21.15.70:0/233522633 shutdown_connections 2024-09-18T07:13:56.598 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.595+0000 7f546b510640 1 -- 172.21.15.70:0/233522633 >> 172.21.15.70:0/233522633 conn(0x7f54640b3b40 msgr2=0x7f54640afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:13:56.598 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.595+0000 7f546b510640 1 -- 172.21.15.70:0/233522633 shutdown_connections 2024-09-18T07:13:56.598 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:13:56.595+0000 7f546b510640 1 -- 172.21.15.70:0/233522633 wait complete. 2024-09-18T07:13:56.607 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-18T07:13:56.653 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-18T07:13:56.650+0000 7fb315415640 -1 Falling back to public interface 2024-09-18T07:13:56.928 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-18T07:13:56.925+0000 7f47e5813640 -1 Falling back to public interface 2024-09-18T07:13:57.140 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-18T07:13:57.137+0000 7fd93131e640 -1 Falling back to public interface 2024-09-18T07:13:57.174 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-18T07:13:57.171+0000 7f744c21c640 -1 Falling back to public interface 2024-09-18T07:13:57.185 INFO:tasks.ceph.osd.5.smithi172.stderr:2024-09-18T07:13:57.183+0000 7fa4fff45640 -1 Falling back to public interface 2024-09-18T07:13:57.187 INFO:tasks.ceph.osd.4.smithi172.stderr:2024-09-18T07:13:57.185+0000 7f3a09644640 -1 Falling back to public interface 2024-09-18T07:13:57.192 INFO:tasks.ceph.osd.6.smithi172.stderr:2024-09-18T07:13:57.190+0000 7f1ff840d640 -1 Falling back to public interface 2024-09-18T07:13:58.453 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-18T07:13:58.449+0000 7f0227e2f640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-18T07:13:58.453 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-18T07:13:58.450+0000 7f0227e2f640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-18T07:14:00.489 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-18T07:14:00.486+0000 7f47e5813640 -1 osd.3 0 log_to_monitors true 2024-09-18T07:14:00.499 INFO:tasks.ceph.osd.7.smithi172.stderr:2024-09-18T07:14:00.496+0000 7fa658c0f640 -1 osd.7 0 log_to_monitors true 2024-09-18T07:14:00.707 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-18T07:14:00.705+0000 7fb315415640 -1 osd.1 0 log_to_monitors true 2024-09-18T07:14:00.753 INFO:tasks.ceph.osd.6.smithi172.stderr:2024-09-18T07:14:00.750+0000 7f1ff840d640 -1 osd.6 0 log_to_monitors true 2024-09-18T07:14:00.887 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-18T07:14:00.884+0000 7fd93131e640 -1 osd.0 0 log_to_monitors true 2024-09-18T07:14:00.936 INFO:tasks.ceph.osd.5.smithi172.stderr:2024-09-18T07:14:00.933+0000 7fa4fff45640 -1 osd.5 0 log_to_monitors true 2024-09-18T07:14:01.428 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-18T07:14:01.426+0000 7f744c21c640 -1 osd.2 0 log_to_monitors true 2024-09-18T07:14:01.720 INFO:tasks.ceph.osd.4.smithi172.stderr:2024-09-18T07:14:01.716+0000 7f3a09644640 -1 osd.4 0 log_to_monitors true 2024-09-18T07:14:02.486 INFO:tasks.ceph.osd.6.smithi172.stderr:2024-09-18T07:14:02.483+0000 7f1ff3e4b640 -1 osd.6 0 waiting for initial osdmap 2024-09-18T07:14:02.486 INFO:tasks.ceph.osd.7.smithi172.stderr:2024-09-18T07:14:02.483+0000 7fa6544ed640 -1 osd.7 0 waiting for initial osdmap 2024-09-18T07:14:02.489 INFO:tasks.ceph.osd.7.smithi172.stderr:2024-09-18T07:14:02.487+0000 7fa650314640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T07:14:02.491 INFO:tasks.ceph.osd.6.smithi172.stderr:2024-09-18T07:14:02.488+0000 7f1fefc72640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T07:14:03.212 INFO:tasks.ceph.osd.5.smithi172.stderr:2024-09-18T07:14:03.209+0000 7fa4fbed4640 -1 osd.5 0 waiting for initial osdmap 2024-09-18T07:14:03.214 INFO:tasks.ceph.osd.5.smithi172.stderr:2024-09-18T07:14:03.211+0000 7fa4f3cf3640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T07:14:03.415 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T07:14:03.466 INFO:tasks.ceph.osd.4.smithi172.stderr:2024-09-18T07:14:03.463+0000 7f3a055d3640 -1 osd.4 0 waiting for initial osdmap 2024-09-18T07:14:03.469 INFO:tasks.ceph.osd.4.smithi172.stderr:2024-09-18T07:14:03.466+0000 7f3a013fa640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T07:14:03.573 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.570+0000 7fdbbc8d3640 1 Processor -- start 2024-09-18T07:14:03.573 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.570+0000 7fdbbc8d3640 1 -- start start 2024-09-18T07:14:03.574 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.571+0000 7fdbbc8d3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb0032420 con 0x7fdbb0006b70 2024-09-18T07:14:03.575 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.571+0000 7fdbbc8d3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb0032700 con 0x7fdbb00caee0 2024-09-18T07:14:03.575 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.571+0000 7fdbbc8d3640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb00329e0 con 0x7fdbb0036b90 2024-09-18T07:14:03.575 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.572+0000 7fdbb659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdbb0006b70 0x7fdbb00ca7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:56172/0 (socket says 172.21.15.70:56172) 2024-09-18T07:14:03.575 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.572+0000 7fdbb659c640 1 -- 172.21.15.70:0/688721395 learned_addr learned my addr 172.21.15.70:0/688721395 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:03.575 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.572+0000 7fdbb5d9b640 1 -- 172.21.15.70:0/688721395 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 392503426 0 0) 0x7fdbb0032420 con 0x7fdbb0006b70 2024-09-18T07:14:03.575 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.572+0000 7fdbb5d9b640 1 -- 172.21.15.70:0/688721395 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdba0003520 con 0x7fdbb0006b70 2024-09-18T07:14:03.576 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.572+0000 7fdbb5d9b640 1 -- 172.21.15.70:0/688721395 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2241055943 0 0) 0x7fdba0003520 con 0x7fdbb0006b70 2024-09-18T07:14:03.576 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.572+0000 7fdbb5d9b640 1 -- 172.21.15.70:0/688721395 >> v1:172.21.15.70:6790/0 conn(0x7fdbb0036b90 legacy=0x7fdbb0032e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-18T07:14:03.576 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.573+0000 7fdbb5d9b640 1 -- 172.21.15.70:0/688721395 >> v1:172.21.15.172:6789/0 conn(0x7fdbb00caee0 legacy=0x7fdbb00d6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:03.576 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.573+0000 7fdbb5d9b640 1 -- 172.21.15.70:0/688721395 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb0033580 con 0x7fdbb0006b70 2024-09-18T07:14:03.576 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.573+0000 7fdbb5d9b640 1 -- 172.21.15.70:0/688721395 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fdba8001690 con 0x7fdbb0006b70 2024-09-18T07:14:03.577 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.573+0000 7fdbb5d9b640 1 -- 172.21.15.70:0/688721395 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdba8001960 con 0x7fdbb0006b70 2024-09-18T07:14:03.577 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.573+0000 7fdbb5d9b640 1 -- 172.21.15.70:0/688721395 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fdba8001d30 con 0x7fdbb0006b70 2024-09-18T07:14:03.577 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.574+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/688721395 >> v1:172.21.15.70:6789/0 conn(0x7fdbb0006b70 legacy=0x7fdbb00ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:03.577 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.575+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/688721395 shutdown_connections 2024-09-18T07:14:03.577 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.575+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/688721395 >> 172.21.15.70:0/688721395 conn(0x7fdbb0039d60 msgr2=0x7fdbb00361b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:03.578 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.575+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/688721395 shutdown_connections 2024-09-18T07:14:03.580 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.576+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/688721395 wait complete. 2024-09-18T07:14:03.580 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.577+0000 7fdbbc8d3640 1 Processor -- start 2024-09-18T07:14:03.582 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.578+0000 7fdbbc8d3640 1 -- start start 2024-09-18T07:14:03.582 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.579+0000 7fdbbc8d3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb00a2b90 con 0x7fdbb00caee0 2024-09-18T07:14:03.582 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.579+0000 7fdbbc8d3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb00a1ce0 con 0x7fdbb0006b70 2024-09-18T07:14:03.582 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.579+0000 7fdbbc8d3640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb00c9e60 con 0x7fdbb0036b90 2024-09-18T07:14:03.582 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.579+0000 7fdbb659c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fdbb0036b90 0x7fdbb00c9040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:39926/0 (socket says 172.21.15.70:39926) 2024-09-18T07:14:03.583 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.579+0000 7fdbb659c640 1 -- 172.21.15.70:0/3066688564 learned_addr learned my addr 172.21.15.70:0/3066688564 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:03.583 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.579+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4155025472 0 0) 0x7fdbb00c9e60 con 0x7fdbb0036b90 2024-09-18T07:14:03.583 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.579+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdba4003520 con 0x7fdbb0036b90 2024-09-18T07:14:03.583 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.579+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2556048870 0 0) 0x7fdbb00a2b90 con 0x7fdbb00caee0 2024-09-18T07:14:03.583 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.579+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbb00c9e60 con 0x7fdbb00caee0 2024-09-18T07:14:03.584 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.580+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3460000101 0 0) 0x7fdba4003520 con 0x7fdbb0036b90 2024-09-18T07:14:03.584 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.580+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdbb00a2b90 con 0x7fdbb0036b90 2024-09-18T07:14:03.584 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.580+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fdba8001550 con 0x7fdbb0036b90 2024-09-18T07:14:03.584 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.580+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2136191806 0 0) 0x7fdbb00c9e60 con 0x7fdbb00caee0 2024-09-18T07:14:03.584 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.580+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdba4003520 con 0x7fdbb00caee0 2024-09-18T07:14:03.584 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.580+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fdbac001790 con 0x7fdbb00caee0 2024-09-18T07:14:03.584 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.581+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3472969513 0 0) 0x7fdba4003520 con 0x7fdbb00caee0 2024-09-18T07:14:03.585 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.581+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 >> v1:172.21.15.70:6790/0 conn(0x7fdbb0036b90 legacy=0x7fdbb00c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:03.585 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.581+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 >> v1:172.21.15.172:6789/0 conn(0x7fdbb0006b70 legacy=0x7fdbb016c2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-18T07:14:03.585 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.581+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb00a1370 con 0x7fdbb00caee0 2024-09-18T07:14:03.585 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.581+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/3066688564 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb002f8d0 con 0x7fdbb00caee0 2024-09-18T07:14:03.585 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.581+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/3066688564 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fdbb002fc60 con 0x7fdbb00caee0 2024-09-18T07:14:03.585 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.581+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdbac001c60 con 0x7fdbb00caee0 2024-09-18T07:14:03.585 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.581+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fdbac0015f0 con 0x7fdbb00caee0 2024-09-18T07:14:03.588 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.584+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/3066688564 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb98005080 con 0x7fdbb00caee0 2024-09-18T07:14:03.588 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.584+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 4) ==== 101350+0+0 (unknown 1995015620 0 0) 0x7fdbac01c280 con 0x7fdbb00caee0 2024-09-18T07:14:03.589 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.585+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(13..13 src has 1..13) ==== 2651+0+0 (unknown 771417106 0 0) 0x7fdbac094030 con 0x7fdbb00caee0 2024-09-18T07:14:03.594 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.592+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7fdbac05d060 con 0x7fdbb00caee0 2024-09-18T07:14:03.827 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-18T07:14:03.824+0000 7fb310d26640 -1 osd.1 0 waiting for initial osdmap 2024-09-18T07:14:03.827 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-18T07:14:03.825+0000 7f74481ab640 -1 osd.2 0 waiting for initial osdmap 2024-09-18T07:14:03.832 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-18T07:14:03.830+0000 7fb308b45640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T07:14:03.836 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-18T07:14:03.833+0000 7f743ffca640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T07:14:03.869 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-18T07:14:03.866+0000 7f47e1106640 -1 osd.3 0 waiting for initial osdmap 2024-09-18T07:14:03.872 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-18T07:14:03.870+0000 7f47d8f25640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T07:14:03.912 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-18T07:14:03.909+0000 7fd92d2ad640 -1 osd.0 0 waiting for initial osdmap 2024-09-18T07:14:03.915 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-18T07:14:03.912+0000 7fd9250cc640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T07:14:03.918 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.916+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/3066688564 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb98005370 con 0x7fdbb00caee0 2024-09-18T07:14:03.919 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.916+0000 7fdb9f7fe640 1 -- 172.21.15.70:0/3066688564 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7799 (unknown 62554612 0 1431338937) 0x7fdbac05d060 con 0x7fdbb00caee0 2024-09-18T07:14:03.919 INFO:teuthology.misc.health.smithi070.stdout: 2024-09-18T07:14:03.919 INFO:teuthology.misc.health.smithi070.stdout:{"epoch":13,"fsid":"1fa528ad-1630-4e95-9001-978a53f2a42c","created":"2024-09-18T07:13:53.234326+0000","modified":"2024-09-18T07:14:03.458299+0000","last_up_change":"2024-09-18T07:14:03.458299+0000","last_in_change":"2024-09-18T07:13:55.403239+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a4a2c5fa-37b8-4776-bf81-f142248da0da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3fab2225-9881-4caf-8142-41b857b9b99a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5d6764cd-e7a4-45df-9cfb-88d9e9a2675f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"80961d78-b7ae-416d-bb6f-a2ef724c6cff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ac72b675-4df6-4668-8461-3d0d8ac698b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5949328d-408f-4b72-8e51-db0046fbb090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6804","nonce":3053548626}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6805","nonce":3053548626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6807","nonce":3053548626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6806","nonce":3053548626}]},"public_addr":"172.21.15.172:6804/3053548626","cluster_addr":"172.21.15.172:6805/3053548626","heartbeat_back_addr":"172.21.15.172:6807/3053548626","heartbeat_front_addr":"172.21.15.172:6806/3053548626","state":["exists","up"]},{"osd":6,"uuid":"84ce0e0b-9072-4ef7-9e55-f3d81283b3de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6812","nonce":371905845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6813","nonce":371905845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6815","nonce":371905845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6814","nonce":371905845}]},"public_addr":"172.21.15.172:6812/371905845","cluster_addr":"172.21.15.172:6813/371905845","heartbeat_back_addr":"172.21.15.172:6815/371905845","heartbeat_front_addr":"172.21.15.172:6814/371905845","state":["exists","up"]},{"osd":7,"uuid":"96889def-b533-4738-bb6d-dd8eb99e1e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6800","nonce":2423240725}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6801","nonce":2423240725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6803","nonce":2423240725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6802","nonce":2423240725}]},"public_addr":"172.21.15.172:6800/2423240725","cluster_addr":"172.21.15.172:6801/2423240725","heartbeat_back_addr":"172.21.15.172:6803/2423240725","heartbeat_front_addr":"172.21.15.172:6802/2423240725","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T07:14:03.921 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.918+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/3066688564 >> v1:172.21.15.70:6816/1657990791 conn(0x7fdba4079de0 legacy=0x7fdba407c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:03.921 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.918+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/3066688564 >> v1:172.21.15.70:6789/0 conn(0x7fdbb00caee0 legacy=0x7fdbb00c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:03.921 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.918+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/3066688564 shutdown_connections 2024-09-18T07:14:03.921 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.918+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/3066688564 >> 172.21.15.70:0/3066688564 conn(0x7fdbb0039d60 msgr2=0x7fdbb00ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:03.921 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.918+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/3066688564 shutdown_connections 2024-09-18T07:14:03.921 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:03.918+0000 7fdbbc8d3640 1 -- 172.21.15.70:0/3066688564 wait complete. 2024-09-18T07:14:03.930 DEBUG:teuthology.misc:3 of 8 OSDs are up 2024-09-18T07:14:10.737 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T07:14:10.838 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.833+0000 7fc53a782640 1 Processor -- start 2024-09-18T07:14:10.838 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.833+0000 7fc53a782640 1 -- start start 2024-09-18T07:14:10.838 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.834+0000 7fc53a782640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc534073610 con 0x7fc534071b30 2024-09-18T07:14:10.838 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.834+0000 7fc53a782640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5340738f0 con 0x7fc534074a20 2024-09-18T07:14:10.838 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.834+0000 7fc53a782640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5340729d0 con 0x7fc5340af8d0 2024-09-18T07:14:10.838 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.834+0000 7fc533fff640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fc5340af8d0 0x7fc534074310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:39962/0 (socket says 172.21.15.70:39962) 2024-09-18T07:14:10.838 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.834+0000 7fc533fff640 1 -- 172.21.15.70:0/2675263773 learned_addr learned my addr 172.21.15.70:0/2675263773 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:10.840 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.834+0000 7fc5337fe640 1 -- 172.21.15.70:0/2675263773 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1819552 0 0) 0x7fc5340729d0 con 0x7fc5340af8d0 2024-09-18T07:14:10.840 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.835+0000 7fc5337fe640 1 -- 172.21.15.70:0/2675263773 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc520003520 con 0x7fc5340af8d0 2024-09-18T07:14:10.840 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.835+0000 7fc5337fe640 1 -- 172.21.15.70:0/2675263773 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 343247745 0 0) 0x7fc534073610 con 0x7fc534071b30 2024-09-18T07:14:10.840 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.835+0000 7fc5337fe640 1 -- 172.21.15.70:0/2675263773 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5340729d0 con 0x7fc534071b30 2024-09-18T07:14:10.840 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.835+0000 7fc5337fe640 1 -- 172.21.15.70:0/2675263773 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 573974955 0 0) 0x7fc520003520 con 0x7fc5340af8d0 2024-09-18T07:14:10.840 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.835+0000 7fc5337fe640 1 -- 172.21.15.70:0/2675263773 >> v1:172.21.15.172:6789/0 conn(0x7fc534074a20 legacy=0x7fc53413d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:10.840 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.835+0000 7fc5337fe640 1 -- 172.21.15.70:0/2675263773 >> v1:172.21.15.70:6789/0 conn(0x7fc534071b30 legacy=0x7fc53414b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:10.840 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.835+0000 7fc5337fe640 1 -- 172.21.15.70:0/2675263773 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5341434c0 con 0x7fc5340af8d0 2024-09-18T07:14:10.841 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.835+0000 7fc5337fe640 1 -- 172.21.15.70:0/2675263773 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc524001690 con 0x7fc5340af8d0 2024-09-18T07:14:10.841 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.835+0000 7fc5337fe640 1 -- 172.21.15.70:0/2675263773 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc524001830 con 0x7fc5340af8d0 2024-09-18T07:14:10.841 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.835+0000 7fc5337fe640 1 -- 172.21.15.70:0/2675263773 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc524001cc0 con 0x7fc5340af8d0 2024-09-18T07:14:10.841 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.835+0000 7fc53a782640 1 -- 172.21.15.70:0/2675263773 >> v1:172.21.15.70:6790/0 conn(0x7fc5340af8d0 legacy=0x7fc534074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:10.841 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.836+0000 7fc53a782640 1 -- 172.21.15.70:0/2675263773 shutdown_connections 2024-09-18T07:14:10.841 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.836+0000 7fc53a782640 1 -- 172.21.15.70:0/2675263773 >> 172.21.15.70:0/2675263773 conn(0x7fc5340b4c60 msgr2=0x7fc5340b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:10.841 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.836+0000 7fc53a782640 1 -- 172.21.15.70:0/2675263773 shutdown_connections 2024-09-18T07:14:10.841 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.836+0000 7fc53a782640 1 -- 172.21.15.70:0/2675263773 wait complete. 2024-09-18T07:14:10.841 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.837+0000 7fc53a782640 1 Processor -- start 2024-09-18T07:14:10.841 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.837+0000 7fc53a782640 1 -- start start 2024-09-18T07:14:10.844 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.837+0000 7fc53a782640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5341399c0 con 0x7fc534071b30 2024-09-18T07:14:10.844 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.837+0000 7fc53a782640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc534131a90 con 0x7fc534074a20 2024-09-18T07:14:10.844 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.837+0000 7fc53a782640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc534131c60 con 0x7fc5340af8d0 2024-09-18T07:14:10.845 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.837+0000 7fc533fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc534071b30 0x7fc5341d63a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:56232/0 (socket says 172.21.15.70:56232) 2024-09-18T07:14:10.845 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.837+0000 7fc533fff640 1 -- 172.21.15.70:0/3384199567 learned_addr learned my addr 172.21.15.70:0/3384199567 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:10.845 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.837+0000 7fc538d1f640 1 --1- 172.21.15.70:0/3384199567 >> v1:172.21.15.70:6790/0 conn(0x7fc5340af8d0 0x7fc5341e0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:39966/0 (socket says 172.21.15.70:39966) 2024-09-18T07:14:10.845 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.837+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 214807623 0 0) 0x7fc5341399c0 con 0x7fc534071b30 2024-09-18T07:14:10.845 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.837+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc514003550 con 0x7fc534071b30 2024-09-18T07:14:10.845 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.837+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2924342955 0 0) 0x7fc534131c60 con 0x7fc5340af8d0 2024-09-18T07:14:10.845 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.837+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5341399c0 con 0x7fc5340af8d0 2024-09-18T07:14:10.845 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1215334643 0 0) 0x7fc534131a90 con 0x7fc534074a20 2024-09-18T07:14:10.845 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc534131c60 con 0x7fc534074a20 2024-09-18T07:14:10.845 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 607033671 0 0) 0x7fc514003550 con 0x7fc534071b30 2024-09-18T07:14:10.845 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc534131a90 con 0x7fc534071b30 2024-09-18T07:14:10.845 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc524001b80 con 0x7fc534071b30 2024-09-18T07:14:10.846 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1065800225 0 0) 0x7fc5341399c0 con 0x7fc5340af8d0 2024-09-18T07:14:10.846 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc514003550 con 0x7fc5340af8d0 2024-09-18T07:14:10.846 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc52c001b50 con 0x7fc5340af8d0 2024-09-18T07:14:10.846 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1129851993 0 0) 0x7fc534131c60 con 0x7fc534074a20 2024-09-18T07:14:10.846 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc5341399c0 con 0x7fc534074a20 2024-09-18T07:14:10.846 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc51c001870 con 0x7fc534074a20 2024-09-18T07:14:10.846 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 139365694 0 0) 0x7fc534131a90 con 0x7fc534071b30 2024-09-18T07:14:10.846 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 >> v1:172.21.15.70:6790/0 conn(0x7fc5340af8d0 legacy=0x7fc5341e0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:10.846 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 >> v1:172.21.15.172:6789/0 conn(0x7fc534074a20 legacy=0x7fc5341d6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:10.846 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc534133ac0 con 0x7fc534071b30 2024-09-18T07:14:10.846 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc53a782640 1 -- 172.21.15.70:0/3384199567 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc534133db0 con 0x7fc534071b30 2024-09-18T07:14:10.846 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc53a782640 1 -- 172.21.15.70:0/3384199567 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc5341385f0 con 0x7fc534071b30 2024-09-18T07:14:10.846 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc524004bf0 con 0x7fc534071b30 2024-09-18T07:14:10.847 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.838+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc524004da0 con 0x7fc534071b30 2024-09-18T07:14:10.847 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.839+0000 7fc53a782640 1 -- 172.21.15.70:0/3384199567 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc518005080 con 0x7fc534071b30 2024-09-18T07:14:10.847 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.840+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fc52401d9d0 con 0x7fc534071b30 2024-09-18T07:14:10.847 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.840+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(17..17 src has 1..17) ==== 3737+0+0 (unknown 1158936723 0 0) 0x7fc524095c10 con 0x7fc534071b30 2024-09-18T07:14:10.847 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:10.844+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7fc534133db0 con 0x7fc534071b30 2024-09-18T07:14:11.023 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:11.020+0000 7fc53a782640 1 -- 172.21.15.70:0/3384199567 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc518005370 con 0x7fc534071b30 2024-09-18T07:14:11.024 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:11.020+0000 7fc5317fa640 1 -- 172.21.15.70:0/3384199567 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+10968 (unknown 3225238715 0 2170322049) 0x7fc52405e780 con 0x7fc534071b30 2024-09-18T07:14:11.024 INFO:teuthology.misc.health.smithi070.stdout: 2024-09-18T07:14:11.024 INFO:teuthology.misc.health.smithi070.stdout:{"epoch":17,"fsid":"1fa528ad-1630-4e95-9001-978a53f2a42c","created":"2024-09-18T07:13:53.234326+0000","modified":"2024-09-18T07:14:07.489867+0000","last_up_change":"2024-09-18T07:14:04.459974+0000","last_in_change":"2024-09-18T07:13:55.403239+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T07:14:05.461421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4a2c5fa-37b8-4776-bf81-f142248da0da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":1022808125}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":1022808125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":1022808125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":1022808125}]},"public_addr":"172.21.15.70:6808/1022808125","cluster_addr":"172.21.15.70:6809/1022808125","heartbeat_back_addr":"172.21.15.70:6811/1022808125","heartbeat_front_addr":"172.21.15.70:6810/1022808125","state":["exists","up"]},{"osd":1,"uuid":"3fab2225-9881-4caf-8142-41b857b9b99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":1678886977}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":1678886977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":1678886977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":1678886977}]},"public_addr":"172.21.15.70:6800/1678886977","cluster_addr":"172.21.15.70:6801/1678886977","heartbeat_back_addr":"172.21.15.70:6803/1678886977","heartbeat_front_addr":"172.21.15.70:6802/1678886977","state":["exists","up"]},{"osd":2,"uuid":"5d6764cd-e7a4-45df-9cfb-88d9e9a2675f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":3042930971}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":3042930971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":3042930971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":3042930971}]},"public_addr":"172.21.15.70:6812/3042930971","cluster_addr":"172.21.15.70:6813/3042930971","heartbeat_back_addr":"172.21.15.70:6815/3042930971","heartbeat_front_addr":"172.21.15.70:6814/3042930971","state":["exists","up"]},{"osd":3,"uuid":"80961d78-b7ae-416d-bb6f-a2ef724c6cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":202753903}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":202753903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":202753903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":202753903}]},"public_addr":"172.21.15.70:6804/202753903","cluster_addr":"172.21.15.70:6805/202753903","heartbeat_back_addr":"172.21.15.70:6807/202753903","heartbeat_front_addr":"172.21.15.70:6806/202753903","state":["exists","up"]},{"osd":4,"uuid":"ac72b675-4df6-4668-8461-3d0d8ac698b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6808","nonce":3405680289}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6809","nonce":3405680289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6811","nonce":3405680289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6810","nonce":3405680289}]},"public_addr":"172.21.15.172:6808/3405680289","cluster_addr":"172.21.15.172:6809/3405680289","heartbeat_back_addr":"172.21.15.172:6811/3405680289","heartbeat_front_addr":"172.21.15.172:6810/3405680289","state":["exists","up"]},{"osd":5,"uuid":"5949328d-408f-4b72-8e51-db0046fbb090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6804","nonce":3053548626}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6805","nonce":3053548626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6807","nonce":3053548626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6806","nonce":3053548626}]},"public_addr":"172.21.15.172:6804/3053548626","cluster_addr":"172.21.15.172:6805/3053548626","heartbeat_back_addr":"172.21.15.172:6807/3053548626","heartbeat_front_addr":"172.21.15.172:6806/3053548626","state":["exists","up"]},{"osd":6,"uuid":"84ce0e0b-9072-4ef7-9e55-f3d81283b3de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6812","nonce":371905845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6813","nonce":371905845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6815","nonce":371905845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6814","nonce":371905845}]},"public_addr":"172.21.15.172:6812/371905845","cluster_addr":"172.21.15.172:6813/371905845","heartbeat_back_addr":"172.21.15.172:6815/371905845","heartbeat_front_addr":"172.21.15.172:6814/371905845","state":["exists","up"]},{"osd":7,"uuid":"96889def-b533-4738-bb6d-dd8eb99e1e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6800","nonce":2423240725}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6801","nonce":2423240725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6803","nonce":2423240725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6802","nonce":2423240725}]},"public_addr":"172.21.15.172:6800/2423240725","cluster_addr":"172.21.15.172:6801/2423240725","heartbeat_back_addr":"172.21.15.172:6803/2423240725","heartbeat_front_addr":"172.21.15.172:6802/2423240725","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.866556+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.704725+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:02.409291+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.477089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:02.747153+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.980019+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.715388+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.500717+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T07:14:11.026 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:11.022+0000 7fc53a782640 1 -- 172.21.15.70:0/3384199567 >> v1:172.21.15.70:6816/1657990791 conn(0x7fc5140799e0 legacy=0x7fc51407bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:11.026 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:11.022+0000 7fc53a782640 1 -- 172.21.15.70:0/3384199567 >> v1:172.21.15.70:6789/0 conn(0x7fc534071b30 legacy=0x7fc5341d63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:11.026 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:11.023+0000 7fc53a782640 1 -- 172.21.15.70:0/3384199567 shutdown_connections 2024-09-18T07:14:11.026 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:11.023+0000 7fc53a782640 1 -- 172.21.15.70:0/3384199567 >> 172.21.15.70:0/3384199567 conn(0x7fc5340b4c60 msgr2=0x7fc5340afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:11.027 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:11.023+0000 7fc53a782640 1 -- 172.21.15.70:0/3384199567 shutdown_connections 2024-09-18T07:14:11.027 INFO:teuthology.misc.health.smithi070.stderr:2024-09-18T07:14:11.023+0000 7fc53a782640 1 -- 172.21.15.70:0/3384199567 wait complete. 2024-09-18T07:14:11.035 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-18T07:14:11.036 INFO:tasks.ceph:Creating RBD pool 2024-09-18T07:14:11.036 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-18T07:14:11.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.131+0000 7f676e789640 1 Processor -- start 2024-09-18T07:14:11.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.131+0000 7f676e789640 1 -- start start 2024-09-18T07:14:11.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.131+0000 7f676e789640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67680aed30 con 0x7f67680b02d0 2024-09-18T07:14:11.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.131+0000 7f676e789640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67680ad580 con 0x7f67680b06b0 2024-09-18T07:14:11.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.131+0000 7f676e789640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67680ad860 con 0x7f67680ae900 2024-09-18T07:14:11.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.132+0000 7f676cd26640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f67680ae900 0x7f676814b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37450/0 (socket says 172.21.15.70:37450) 2024-09-18T07:14:11.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.132+0000 7f676cd26640 1 -- 172.21.15.70:0/1582587834 learned_addr learned my addr 172.21.15.70:0/1582587834 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:11.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.132+0000 7f6766ffd640 1 -- 172.21.15.70:0/1582587834 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2297001992 0 0) 0x7f67680ad860 con 0x7f67680ae900 2024-09-18T07:14:11.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.132+0000 7f6766ffd640 1 -- 172.21.15.70:0/1582587834 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6754003520 con 0x7f67680ae900 2024-09-18T07:14:11.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.132+0000 7f6766ffd640 1 -- 172.21.15.70:0/1582587834 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2084138288 0 0) 0x7f6758001110 con 0x7f67680b02d0 2024-09-18T07:14:11.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.132+0000 7f6766ffd640 1 -- 172.21.15.70:0/1582587834 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67680ad860 con 0x7f67680b02d0 2024-09-18T07:14:11.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.132+0000 7f6766ffd640 1 -- 172.21.15.70:0/1582587834 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4200064888 0 0) 0x7f6754003520 con 0x7f67680ae900 2024-09-18T07:14:11.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.132+0000 7f6766ffd640 1 -- 172.21.15.70:0/1582587834 >> v1:172.21.15.172:6789/0 conn(0x7f67680b06b0 legacy=0x7f67680ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:11.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.132+0000 7f6766ffd640 1 -- 172.21.15.70:0/1582587834 >> v1:172.21.15.70:6789/0 conn(0x7f67680b02d0 legacy=0x7f676813ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:11.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.132+0000 7f6766ffd640 1 -- 172.21.15.70:0/1582587834 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67680adb40 con 0x7f67680ae900 2024-09-18T07:14:11.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.132+0000 7f6766ffd640 1 -- 172.21.15.70:0/1582587834 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6760001a10 con 0x7f67680ae900 2024-09-18T07:14:11.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.132+0000 7f6766ffd640 1 -- 172.21.15.70:0/1582587834 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6760001bb0 con 0x7f67680ae900 2024-09-18T07:14:11.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.132+0000 7f6766ffd640 1 -- 172.21.15.70:0/1582587834 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6760001fc0 con 0x7f67680ae900 2024-09-18T07:14:11.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.133+0000 7f676e789640 1 -- 172.21.15.70:0/1582587834 >> v1:172.21.15.70:6790/0 conn(0x7f67680ae900 legacy=0x7f676814b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:11.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.133+0000 7f676e789640 1 -- 172.21.15.70:0/1582587834 shutdown_connections 2024-09-18T07:14:11.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.133+0000 7f676e789640 1 -- 172.21.15.70:0/1582587834 >> 172.21.15.70:0/1582587834 conn(0x7f67680b1b70 msgr2=0x7f67680b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:11.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.133+0000 7f676e789640 1 -- 172.21.15.70:0/1582587834 shutdown_connections 2024-09-18T07:14:11.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.133+0000 7f676e789640 1 -- 172.21.15.70:0/1582587834 wait complete. 2024-09-18T07:14:11.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.134+0000 7f676e789640 1 Processor -- start 2024-09-18T07:14:11.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.134+0000 7f676e789640 1 -- start start 2024-09-18T07:14:11.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.134+0000 7f676e789640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6768135240 con 0x7f67680ae900 2024-09-18T07:14:11.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.134+0000 7f676e789640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676813bee0 con 0x7f67680b02d0 2024-09-18T07:14:11.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.134+0000 7f676e789640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6768070a40 con 0x7f67681da700 2024-09-18T07:14:11.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.134+0000 7f6767fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f67680ae900 0x7f6768070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51664/0 (socket says 172.21.15.70:51664) 2024-09-18T07:14:11.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.134+0000 7f6767fff640 1 -- 172.21.15.70:0/4079899423 learned_addr learned my addr 172.21.15.70:0/4079899423 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:11.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 606123213 0 0) 0x7f6768135240 con 0x7f67680ae900 2024-09-18T07:14:11.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6748003520 con 0x7f67680ae900 2024-09-18T07:14:11.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 210059115 0 0) 0x7f6768070a40 con 0x7f67681da700 2024-09-18T07:14:11.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6768135240 con 0x7f67681da700 2024-09-18T07:14:11.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4163497407 0 0) 0x7f676813bee0 con 0x7f67680b02d0 2024-09-18T07:14:11.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6768070a40 con 0x7f67680b02d0 2024-09-18T07:14:11.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2050795317 0 0) 0x7f6748003520 con 0x7f67680ae900 2024-09-18T07:14:11.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f676813bee0 con 0x7f67680ae900 2024-09-18T07:14:11.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2986261069 0 0) 0x7f6768135240 con 0x7f67681da700 2024-09-18T07:14:11.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6748003520 con 0x7f67681da700 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6758001580 con 0x7f67680ae900 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6760001d60 con 0x7f67681da700 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3185262745 0 0) 0x7f6768070a40 con 0x7f67680b02d0 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6768135240 con 0x7f67680b02d0 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f67500017d0 con 0x7f67680b02d0 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2370488495 0 0) 0x7f676813bee0 con 0x7f67680ae900 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 >> v1:172.21.15.70:6790/0 conn(0x7f67681da700 legacy=0x7f67681e1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 >> v1:172.21.15.172:6789/0 conn(0x7f67680b02d0 legacy=0x7f6768130dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67681314d0 con 0x7f67680ae900 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f676e789640 1 -- 172.21.15.70:0/4079899423 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6768073990 con 0x7f67680ae900 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f676e789640 1 -- 172.21.15.70:0/4079899423 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f67681d9f30 con 0x7f67680ae900 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f67580038a0 con 0x7f67680ae900 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.135+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6758003b50 con 0x7f67680ae900 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.136+0000 7f676e789640 1 -- 172.21.15.70:0/4079899423 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f674c005080 con 0x7f67680ae900 2024-09-18T07:14:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.137+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f6758003b10 con 0x7f67680ae900 2024-09-18T07:14:11.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.137+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(17..17 src has 1..17) ==== 3737+0+0 (unknown 1158936723 0 0) 0x7f6758095d70 con 0x7f67680ae900 2024-09-18T07:14:11.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.141+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f67680aed30 con 0x7f67680ae900 2024-09-18T07:14:11.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:11.318+0000 7f676e789640 1 -- 172.21.15.70:0/4079899423 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f674c005c40 con 0x7f67680ae900 2024-09-18T07:14:12.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.236+0000 7f6764ff9640 1 -- 172.21.15.70:0/4079899423 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v18) ==== 109+0+0 (unknown 4122451468 0 0) 0x7f675805e8c0 con 0x7f67680ae900 2024-09-18T07:14:12.240 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'rbd' created 2024-09-18T07:14:12.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.240+0000 7f676e789640 1 -- 172.21.15.70:0/4079899423 >> v1:172.21.15.70:6816/1657990791 conn(0x7f6748079a10 legacy=0x7f674807bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:12.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.240+0000 7f676e789640 1 -- 172.21.15.70:0/4079899423 >> v1:172.21.15.70:6789/0 conn(0x7f67680ae900 legacy=0x7f6768070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:12.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.241+0000 7f676e789640 1 -- 172.21.15.70:0/4079899423 shutdown_connections 2024-09-18T07:14:12.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.241+0000 7f676e789640 1 -- 172.21.15.70:0/4079899423 >> 172.21.15.70:0/4079899423 conn(0x7f67680b1b70 msgr2=0x7f67680afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:12.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.241+0000 7f676e789640 1 -- 172.21.15.70:0/4079899423 shutdown_connections 2024-09-18T07:14:12.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.241+0000 7f676e789640 1 -- 172.21.15.70:0/4079899423 wait complete. 2024-09-18T07:14:12.257 DEBUG:teuthology.orchestra.run.smithi070:> rbd --cluster ceph pool init rbd 2024-09-18T07:14:12.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.305+0000 7f864b032140 1 Processor -- start 2024-09-18T07:14:12.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.306+0000 7f864b032140 1 -- start start 2024-09-18T07:14:12.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.306+0000 7f864b032140 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x5558bceb1d90 con 0x5558bd015cf0 2024-09-18T07:14:12.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.306+0000 7f864b032140 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x5558bcec9410 con 0x5558bd0d9640 2024-09-18T07:14:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.306+0000 7f864b032140 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x5558bce80720 con 0x5558bd0d30f0 2024-09-18T07:14:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.306+0000 7f8649b1d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x5558bd015cf0 0x5558bd0d29e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51678/0 (socket says 172.21.15.70:51678) 2024-09-18T07:14:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.306+0000 7f8649b1d640 1 -- 172.21.15.70:0/470063188 learned_addr learned my addr 172.21.15.70:0/470063188 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.307+0000 7f864931c640 1 -- 172.21.15.70:0/470063188 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2607977070 0 0) 0x5558bceb1d90 con 0x5558bd015cf0 2024-09-18T07:14:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.307+0000 7f864931c640 1 -- 172.21.15.70:0/470063188 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8638001e10 con 0x5558bd015cf0 2024-09-18T07:14:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.307+0000 7f864931c640 1 -- 172.21.15.70:0/470063188 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 285543686 0 0) 0x5558bce80720 con 0x5558bd0d30f0 2024-09-18T07:14:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.307+0000 7f864931c640 1 -- 172.21.15.70:0/470063188 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x5558bceb1d90 con 0x5558bd0d30f0 2024-09-18T07:14:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.307+0000 7f864931c640 1 -- 172.21.15.70:0/470063188 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 710198494 0 0) 0x5558bceb1d90 con 0x5558bd0d30f0 2024-09-18T07:14:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.307+0000 7f864931c640 1 -- 172.21.15.70:0/470063188 >> v1:172.21.15.172:6789/0 conn(0x5558bd0d9640 legacy=0x5558bd0e12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.307+0000 7f864931c640 1 -- 172.21.15.70:0/470063188 >> v1:172.21.15.70:6789/0 conn(0x5558bd015cf0 legacy=0x5558bd0d29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.307+0000 7f864931c640 1 -- 172.21.15.70:0/470063188 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558bd044270 con 0x5558bd0d30f0 2024-09-18T07:14:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.307+0000 7f864931c640 1 -- 172.21.15.70:0/470063188 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8634001670 con 0x5558bd0d30f0 2024-09-18T07:14:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.307+0000 7f864931c640 1 -- 172.21.15.70:0/470063188 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8634001810 con 0x5558bd0d30f0 2024-09-18T07:14:12.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.307+0000 7f864931c640 1 -- 172.21.15.70:0/470063188 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8634001ca0 con 0x5558bd0d30f0 2024-09-18T07:14:12.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.308+0000 7f864b032140 1 -- 172.21.15.70:0/470063188 >> v1:172.21.15.70:6790/0 conn(0x5558bd0d30f0 legacy=0x5558bd0d8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:12.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.308+0000 7f864b032140 1 -- 172.21.15.70:0/470063188 shutdown_connections 2024-09-18T07:14:12.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.308+0000 7f864b032140 1 -- 172.21.15.70:0/470063188 >> 172.21.15.70:0/470063188 conn(0x5558bd0453f0 msgr2=0x5558bd016b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:12.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.308+0000 7f864b032140 1 -- 172.21.15.70:0/470063188 shutdown_connections 2024-09-18T07:14:12.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.308+0000 7f864b032140 1 -- 172.21.15.70:0/470063188 wait complete. 2024-09-18T07:14:12.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.309+0000 7f864b032140 1 Processor -- start 2024-09-18T07:14:12.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.309+0000 7f864b032140 1 -- start start 2024-09-18T07:14:12.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.309+0000 7f864b032140 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x5558bd0cee90 con 0x5558bd015cf0 2024-09-18T07:14:12.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.309+0000 7f864b032140 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x5558bd040500 con 0x5558bd0d30f0 2024-09-18T07:14:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.309+0000 7f864b032140 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x5558bd0d60b0 con 0x5558bd0d9640 2024-09-18T07:14:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.309+0000 7f8649b1d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x5558bd015cf0 0x5558bd0d37b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51688/0 (socket says 172.21.15.70:51688) 2024-09-18T07:14:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.309+0000 7f8649b1d640 1 -- 172.21.15.70:0/1857915150 learned_addr learned my addr 172.21.15.70:0/1857915150 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3715860376 0 0) 0x5558bd0d60b0 con 0x5558bd0d9640 2024-09-18T07:14:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f862c001e10 con 0x5558bd0d9640 2024-09-18T07:14:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3198776009 0 0) 0x5558bd0cee90 con 0x5558bd015cf0 2024-09-18T07:14:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x5558bd0d60b0 con 0x5558bd015cf0 2024-09-18T07:14:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 295560408 0 0) 0x5558bd0d60b0 con 0x5558bd015cf0 2024-09-18T07:14:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x5558bd0cee90 con 0x5558bd015cf0 2024-09-18T07:14:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f863c001390 con 0x5558bd015cf0 2024-09-18T07:14:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3544974799 0 0) 0x7f862c001e10 con 0x5558bd0d9640 2024-09-18T07:14:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x5558bd0d60b0 con 0x5558bd0d9640 2024-09-18T07:14:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8644001a90 con 0x5558bd0d9640 2024-09-18T07:14:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4042747276 0 0) 0x5558bd040500 con 0x5558bd0d30f0 2024-09-18T07:14:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f862c001e10 con 0x5558bd0d30f0 2024-09-18T07:14:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1896914786 0 0) 0x5558bd0cee90 con 0x5558bd015cf0 2024-09-18T07:14:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 >> v1:172.21.15.70:6790/0 conn(0x5558bd0d9640 legacy=0x5558bd0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 >> v1:172.21.15.172:6789/0 conn(0x5558bd0d30f0 legacy=0x5558bd0ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558bd0d65d0 con 0x5558bd015cf0 2024-09-18T07:14:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x5558bd03f2a0 con 0x5558bd015cf0 2024-09-18T07:14:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x5558bd0d40f0 con 0x5558bd015cf0 2024-09-18T07:14:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f863c0030d0 con 0x5558bd015cf0 2024-09-18T07:14:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.310+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f863c003420 con 0x5558bd015cf0 2024-09-18T07:14:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.312+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f863c01c270 con 0x5558bd015cf0 2024-09-18T07:14:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.312+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(18..18 src has 1..18) ==== 4079+0+0 (unknown 467998432 0 0) 0x7f863c0949c0 con 0x5558bd015cf0 2024-09-18T07:14:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:12.313+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) -- 0x5558bd0d85e0 con 0x5558bd015cf0 2024-09-18T07:14:13.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:13.238+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v19) ==== 141+0+0 (unknown 3812259036 0 0) 0x7f863c05d3c0 con 0x5558bd015cf0 2024-09-18T07:14:13.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:13.239+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f8634001810 con 0x5558bd015cf0 2024-09-18T07:14:13.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:13.243+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (unknown 4138596325 0 0) 0x7f863c058ad0 con 0x5558bd015cf0 2024-09-18T07:14:13.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:13.243+0000 7f8648b1b640 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=19}) -- 0x7f8628002ae0 con 0x5558bd015cf0 2024-09-18T07:14:13.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:13.244+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 11 ==== osd_map(19..19 src has 1..19) ==== 666+0+0 (unknown 113974723 0 0) 0x7f863c093640 con 0x5558bd015cf0 2024-09-18T07:14:13.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:13.244+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.172:6804/3053548626 -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19) -- 0x5558bd0f5700 con 0x5558bd0f1bf0 2024-09-18T07:14:13.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:13.250+0000 7f864ad7f640 1 -- 172.21.15.70:0/1857915150 <== osd.5 v1:172.21.15.172:6804/3053548626 1 ==== osd_op_reply(1 rbd_trash [create] v19'1 uv1 ondisk = 0) ==== 153+0+0 (unknown 3947355900 0 0) 0x7f8644003180 con 0x5558bd0f1bf0 2024-09-18T07:14:13.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:13.253+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.172:6804/3053548626 -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19) -- 0x5558bd147f00 con 0x5558bd0f1bf0 2024-09-18T07:14:13.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:13.253+0000 7f864ad7f640 1 -- 172.21.15.70:0/1857915150 <== osd.5 v1:172.21.15.172:6804/3053548626 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (unknown 4146531264 0 0) 0x7f8644003180 con 0x5558bd0f1bf0 2024-09-18T07:14:13.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:13.253+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.172:6804/3053548626 -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19) -- 0x5558bd146ef0 con 0x5558bd0f1bf0 2024-09-18T07:14:13.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:13.253+0000 7f864ad7f640 1 -- 172.21.15.70:0/1857915150 <== osd.5 v1:172.21.15.172:6804/3053548626 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (unknown 4135055820 0 0) 0x7f8644003180 con 0x5558bd0f1bf0 2024-09-18T07:14:13.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:13.253+0000 7f8643fff640 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6789/0 -- pool_op(create unmanaged snap pool 2 tid 4 name v0) -- 0x7f8620001f00 con 0x5558bd015cf0 2024-09-18T07:14:14.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:14.243+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 12 ==== pool_op_reply(tid 4 (0) Success v20) ==== 55+0+0 (unknown 3175065591 0 0) 0x7f863c05ccc0 con 0x5558bd015cf0 2024-09-18T07:14:14.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:14.243+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=20}) -- 0x5558bd0cee90 con 0x5558bd015cf0 2024-09-18T07:14:14.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:14.244+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 13 ==== osd_map(20..20 src has 1..20) ==== 626+0+0 (unknown 625215840 0 0) 0x7f863c066250 con 0x5558bd015cf0 2024-09-18T07:14:14.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:14.244+0000 7f8648b1b640 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.172:6804/3053548626 -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20) -- 0x7f8628005820 con 0x5558bd0f1bf0 2024-09-18T07:14:14.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:14.247+0000 7f864ad7f640 1 -- 172.21.15.70:0/1857915150 <== osd.5 v1:172.21.15.172:6804/3053548626 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v20'1 uv1 ondisk = 0) ==== 194+0+0 (unknown 2229878758 0 0) 0x7f8644003180 con 0x5558bd0f1bf0 2024-09-18T07:14:14.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:14.247+0000 7f8643fff640 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6789/0 -- pool_op(delete unmanaged snap pool 2 tid 6 name v20) -- 0x7f8620003370 con 0x5558bd015cf0 2024-09-18T07:14:15.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.249+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 14 ==== pool_op_reply(tid 6 (0) Success v21) ==== 43+0+0 (unknown 2037885223 0 0) 0x7f863c0617a0 con 0x5558bd015cf0 2024-09-18T07:14:15.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.249+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=21}) -- 0x5558bd040500 con 0x5558bd015cf0 2024-09-18T07:14:15.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.250+0000 7f86427fc640 1 -- 172.21.15.70:0/1857915150 <== mon.0 v1:172.21.15.70:6789/0 15 ==== osd_map(21..21 src has 1..21) ==== 654+0+0 (unknown 2796946614 0 0) 0x7f863c066450 con 0x5558bd015cf0 2024-09-18T07:14:15.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.250+0000 7f8648b1b640 1 -- 172.21.15.70:0/1857915150 --> v1:172.21.15.172:6804/3053548626 -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21) -- 0x7f8628005c00 con 0x5558bd0f1bf0 2024-09-18T07:14:15.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.255+0000 7f864ad7f640 1 -- 172.21.15.70:0/1857915150 <== osd.5 v1:172.21.15.172:6804/3053548626 5 ==== osd_op_reply(7 rbd_info [write 0~19] v21'2 uv2 ondisk = 0) ==== 152+0+0 (unknown 2734904887 0 0) 0x7f8644003180 con 0x5558bd0f1bf0 2024-09-18T07:14:15.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.255+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 >> v1:172.21.15.172:6804/3053548626 conn(0x5558bd0f1bf0 legacy=0x5558bd0f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.255+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 >> v1:172.21.15.70:6816/1657990791 conn(0x7f862c078110 legacy=0x7f862c07a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.255+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 >> v1:172.21.15.70:6789/0 conn(0x5558bd015cf0 legacy=0x5558bd0d37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.256+0000 7f864ad7f640 1 -- 172.21.15.70:0/1857915150 reap_dead start 2024-09-18T07:14:15.259 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.256+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 shutdown_connections 2024-09-18T07:14:15.259 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.256+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 >> 172.21.15.70:0/1857915150 conn(0x5558bd0453f0 msgr2=0x5558bd043320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:15.259 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.256+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 shutdown_connections 2024-09-18T07:14:15.259 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.256+0000 7f864b032140 1 -- 172.21.15.70:0/1857915150 wait complete. 2024-09-18T07:14:15.263 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-18T07:14:15.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-18T07:14:15.264 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-18T07:14:15.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.367+0000 7f42ece48640 1 Processor -- start 2024-09-18T07:14:15.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.367+0000 7f42ece48640 1 -- start start 2024-09-18T07:14:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.367+0000 7f42ece48640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e80ad5d0 con 0x7f42e813fe40 2024-09-18T07:14:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.367+0000 7f42ece48640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e80aecc0 con 0x7f42e8139390 2024-09-18T07:14:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.367+0000 7f42ece48640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e80ad840 con 0x7f42e80b0570 2024-09-18T07:14:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.367+0000 7f42e659c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f42e80b0570 0x7f42e813f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37502/0 (socket says 172.21.15.70:37502) 2024-09-18T07:14:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e659c640 1 -- 172.21.15.70:0/3305042868 learned_addr learned my addr 172.21.15.70:0/3305042868 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.367+0000 7f42ddd9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f42e813fe40 0x7f42e8138c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51690/0 (socket says 172.21.15.70:51690) 2024-09-18T07:14:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e5d9b640 1 -- 172.21.15.70:0/3305042868 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2643072511 0 0) 0x7f42e80ad840 con 0x7f42e80b0570 2024-09-18T07:14:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e5d9b640 1 -- 172.21.15.70:0/3305042868 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42d4003440 con 0x7f42e80b0570 2024-09-18T07:14:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e5d9b640 1 -- 172.21.15.70:0/3305042868 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1552288465 0 0) 0x7f42e80ad5d0 con 0x7f42e813fe40 2024-09-18T07:14:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e5d9b640 1 -- 172.21.15.70:0/3305042868 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42e80ad840 con 0x7f42e813fe40 2024-09-18T07:14:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e5d9b640 1 -- 172.21.15.70:0/3305042868 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2699462073 0 0) 0x7f42e80aecc0 con 0x7f42e8139390 2024-09-18T07:14:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e5d9b640 1 -- 172.21.15.70:0/3305042868 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42e80ad5d0 con 0x7f42e8139390 2024-09-18T07:14:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e5d9b640 1 -- 172.21.15.70:0/3305042868 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2615628938 0 0) 0x7f42d4003440 con 0x7f42e80b0570 2024-09-18T07:14:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e5d9b640 1 -- 172.21.15.70:0/3305042868 >> v1:172.21.15.172:6789/0 conn(0x7f42e8139390 legacy=0x7f42e814b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e5d9b640 1 -- 172.21.15.70:0/3305042868 >> v1:172.21.15.70:6789/0 conn(0x7f42e813fe40 legacy=0x7f42e8138c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e5d9b640 1 -- 172.21.15.70:0/3305042868 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e80adb20 con 0x7f42e80b0570 2024-09-18T07:14:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e5d9b640 1 -- 172.21.15.70:0/3305042868 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f42d8001690 con 0x7f42e80b0570 2024-09-18T07:14:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e5d9b640 1 -- 172.21.15.70:0/3305042868 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f42d8001830 con 0x7f42e80b0570 2024-09-18T07:14:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.368+0000 7f42e5d9b640 1 -- 172.21.15.70:0/3305042868 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f42d8001c40 con 0x7f42e80b0570 2024-09-18T07:14:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.369+0000 7f42ece48640 1 -- 172.21.15.70:0/3305042868 >> v1:172.21.15.70:6790/0 conn(0x7f42e80b0570 legacy=0x7f42e813f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.369+0000 7f42ece48640 1 -- 172.21.15.70:0/3305042868 shutdown_connections 2024-09-18T07:14:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.369+0000 7f42ece48640 1 -- 172.21.15.70:0/3305042868 >> 172.21.15.70:0/3305042868 conn(0x7f42e80b1b70 msgr2=0x7f42e80b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.369+0000 7f42ece48640 1 -- 172.21.15.70:0/3305042868 shutdown_connections 2024-09-18T07:14:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.369+0000 7f42ece48640 1 -- 172.21.15.70:0/3305042868 wait complete. 2024-09-18T07:14:15.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.370+0000 7f42ece48640 1 Processor -- start 2024-09-18T07:14:15.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.370+0000 7f42ece48640 1 -- start start 2024-09-18T07:14:15.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42ece48640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e81d8750 con 0x7f42e813fe40 2024-09-18T07:14:15.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42ece48640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e8140be0 con 0x7f42e80b0570 2024-09-18T07:14:15.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42ece48640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e8143a60 con 0x7f42e8139390 2024-09-18T07:14:15.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42ddd9b640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f42e8139390 0x7f42e8135da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37512/0 (socket says 172.21.15.70:37512) 2024-09-18T07:14:15.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42ddd9b640 1 -- 172.21.15.70:0/1786444692 learned_addr learned my addr 172.21.15.70:0/1786444692 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 9646015 0 0) 0x7f42e8143a60 con 0x7f42e8139390 2024-09-18T07:14:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42c8003520 con 0x7f42e8139390 2024-09-18T07:14:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 617487582 0 0) 0x7f42e81d8750 con 0x7f42e813fe40 2024-09-18T07:14:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42e8143a60 con 0x7f42e813fe40 2024-09-18T07:14:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2954426826 0 0) 0x7f42c8003520 con 0x7f42e8139390 2024-09-18T07:14:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f42e81d8750 con 0x7f42e8139390 2024-09-18T07:14:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3663400956 0 0) 0x7f42e8140be0 con 0x7f42e80b0570 2024-09-18T07:14:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42c8003520 con 0x7f42e80b0570 2024-09-18T07:14:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f42d00018b0 con 0x7f42e8139390 2024-09-18T07:14:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 616796444 0 0) 0x7f42e8143a60 con 0x7f42e813fe40 2024-09-18T07:14:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f42e8140be0 con 0x7f42e813fe40 2024-09-18T07:14:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f42e0001bb0 con 0x7f42e813fe40 2024-09-18T07:14:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1539076317 0 0) 0x7f42e81d8750 con 0x7f42e8139390 2024-09-18T07:14:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 >> v1:172.21.15.172:6789/0 conn(0x7f42e80b0570 legacy=0x7f42e81d8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 >> v1:172.21.15.70:6789/0 conn(0x7f42e813fe40 legacy=0x7f42e81e3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e81d9340 con 0x7f42e8139390 2024-09-18T07:14:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42ece48640 1 -- 172.21.15.70:0/1786444692 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f42e81d6520 con 0x7f42e8139390 2024-09-18T07:14:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42ece48640 1 -- 172.21.15.70:0/1786444692 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f42e813f4e0 con 0x7f42e8139390 2024-09-18T07:14:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.371+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f42d00047e0 con 0x7f42e8139390 2024-09-18T07:14:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.372+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f42d0004b30 con 0x7f42e8139390 2024-09-18T07:14:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.373+0000 7f42ece48640 1 -- 172.21.15.70:0/1786444692 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e80b4430 con 0x7f42e8139390 2024-09-18T07:14:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.373+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f42d001d900 con 0x7f42e8139390 2024-09-18T07:14:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.374+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f42d0096070 con 0x7f42e8139390 2024-09-18T07:14:15.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.378+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f42d005e9e0 con 0x7f42e8139390 2024-09-18T07:14:15.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.555+0000 7f42ece48640 1 -- 172.21.15.70:0/1786444692 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f42e8140600 con 0x7f42e8139390 2024-09-18T07:14:15.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.555+0000 7f42df7fe640 1 -- 172.21.15.70:0/1786444692 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v6) ==== 86+0+159 (unknown 2698850854 0 350383660) 0x7f42d005e2e0 con 0x7f42e8139390 2024-09-18T07:14:15.558 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:14:15.560 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.557+0000 7f42ece48640 1 -- 172.21.15.70:0/1786444692 >> v1:172.21.15.70:6816/1657990791 conn(0x7f42c8079880 legacy=0x7f42c807bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.560 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.557+0000 7f42ece48640 1 -- 172.21.15.70:0/1786444692 >> v1:172.21.15.70:6790/0 conn(0x7f42e8139390 legacy=0x7f42e8135da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.560 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.557+0000 7f42ece48640 1 -- 172.21.15.70:0/1786444692 shutdown_connections 2024-09-18T07:14:15.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.557+0000 7f42ece48640 1 -- 172.21.15.70:0/1786444692 >> 172.21.15.70:0/1786444692 conn(0x7f42e80b1b70 msgr2=0x7f42e80afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:15.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.557+0000 7f42ece48640 1 -- 172.21.15.70:0/1786444692 shutdown_connections 2024-09-18T07:14:15.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.557+0000 7f42ece48640 1 -- 172.21.15.70:0/1786444692 wait complete. 2024-09-18T07:14:15.571 INFO:teuthology.orchestra.run.smithi070.stdout:[{"version":6,"timestamp":"2024-09-18T07:14:03.911083+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"4959.112326"}]}] 2024-09-18T07:14:15.571 INFO:tasks.ceph_manager:config epoch is 6 2024-09-18T07:14:15.571 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-18T07:14:15.572 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-18T07:14:15.572 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-18T07:14:15.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.667+0000 7fd68cb07640 1 Processor -- start 2024-09-18T07:14:15.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.667+0000 7fd68cb07640 1 -- start start 2024-09-18T07:14:15.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd68cb07640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6880ad5d0 con 0x7fd688139390 2024-09-18T07:14:15.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd68cb07640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6880aecc0 con 0x7fd68813fe40 2024-09-18T07:14:15.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd68cb07640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6880ad840 con 0x7fd6880b0570 2024-09-18T07:14:15.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd68659c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fd6880b0570 0x7fd68813f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37528/0 (socket says 172.21.15.70:37528) 2024-09-18T07:14:15.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd68659c640 1 -- 172.21.15.70:0/3889305282 learned_addr learned my addr 172.21.15.70:0/3889305282 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:15.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd685d9b640 1 -- 172.21.15.70:0/3889305282 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3337459145 0 0) 0x7fd6880ad5d0 con 0x7fd688139390 2024-09-18T07:14:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd685d9b640 1 -- 172.21.15.70:0/3889305282 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd674003520 con 0x7fd688139390 2024-09-18T07:14:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd685d9b640 1 -- 172.21.15.70:0/3889305282 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 936079099 0 0) 0x7fd6880ad840 con 0x7fd6880b0570 2024-09-18T07:14:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd685d9b640 1 -- 172.21.15.70:0/3889305282 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6880ad5d0 con 0x7fd6880b0570 2024-09-18T07:14:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd685d9b640 1 -- 172.21.15.70:0/3889305282 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 701980980 0 0) 0x7fd6880aecc0 con 0x7fd68813fe40 2024-09-18T07:14:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd685d9b640 1 -- 172.21.15.70:0/3889305282 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6880ad840 con 0x7fd68813fe40 2024-09-18T07:14:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd685d9b640 1 -- 172.21.15.70:0/3889305282 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2844114249 0 0) 0x7fd674003520 con 0x7fd688139390 2024-09-18T07:14:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd685d9b640 1 -- 172.21.15.70:0/3889305282 >> v1:172.21.15.70:6790/0 conn(0x7fd6880b0570 legacy=0x7fd68813f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd685d9b640 1 -- 172.21.15.70:0/3889305282 >> v1:172.21.15.172:6789/0 conn(0x7fd68813fe40 legacy=0x7fd688138c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd685d9b640 1 -- 172.21.15.70:0/3889305282 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6880adb20 con 0x7fd688139390 2024-09-18T07:14:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd685d9b640 1 -- 172.21.15.70:0/3889305282 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd680001a10 con 0x7fd688139390 2024-09-18T07:14:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd685d9b640 1 -- 172.21.15.70:0/3889305282 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd680001bb0 con 0x7fd688139390 2024-09-18T07:14:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.668+0000 7fd685d9b640 1 -- 172.21.15.70:0/3889305282 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd680001fc0 con 0x7fd688139390 2024-09-18T07:14:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.669+0000 7fd68cb07640 1 -- 172.21.15.70:0/3889305282 >> v1:172.21.15.70:6789/0 conn(0x7fd688139390 legacy=0x7fd68814b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.669+0000 7fd68cb07640 1 -- 172.21.15.70:0/3889305282 shutdown_connections 2024-09-18T07:14:15.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.669+0000 7fd68cb07640 1 -- 172.21.15.70:0/3889305282 >> 172.21.15.70:0/3889305282 conn(0x7fd6880b1b70 msgr2=0x7fd6880b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:15.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.669+0000 7fd68cb07640 1 -- 172.21.15.70:0/3889305282 shutdown_connections 2024-09-18T07:14:15.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.669+0000 7fd68cb07640 1 -- 172.21.15.70:0/3889305282 wait complete. 2024-09-18T07:14:15.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.670+0000 7fd68cb07640 1 Processor -- start 2024-09-18T07:14:15.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.670+0000 7fd68cb07640 1 -- start start 2024-09-18T07:14:15.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd68cb07640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd68813b970 con 0x7fd68813fe40 2024-09-18T07:14:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd68cb07640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6881276f0 con 0x7fd688139390 2024-09-18T07:14:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd68cb07640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd68813a5a0 con 0x7fd6880b0570 2024-09-18T07:14:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd68659c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fd6880b0570 0x7fd68813b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37538/0 (socket says 172.21.15.70:37538) 2024-09-18T07:14:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd68659c640 1 -- 172.21.15.70:0/3550199994 learned_addr learned my addr 172.21.15.70:0/3550199994 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3694855652 0 0) 0x7fd6880adb20 con 0x7fd68813fe40 2024-09-18T07:14:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd668003520 con 0x7fd68813fe40 2024-09-18T07:14:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3676110022 0 0) 0x7fd68813a5a0 con 0x7fd6880b0570 2024-09-18T07:14:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6880adb20 con 0x7fd6880b0570 2024-09-18T07:14:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 490677188 0 0) 0x7fd668003520 con 0x7fd68813fe40 2024-09-18T07:14:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd68813a5a0 con 0x7fd68813fe40 2024-09-18T07:14:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd680001f00 con 0x7fd68813fe40 2024-09-18T07:14:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1722731936 0 0) 0x7fd6880adb20 con 0x7fd6880b0570 2024-09-18T07:14:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd668003520 con 0x7fd6880b0570 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3181483738 0 0) 0x7fd6881276f0 con 0x7fd688139390 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6880adb20 con 0x7fd688139390 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd678001290 con 0x7fd6880b0570 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 323236378 0 0) 0x7fd68813a5a0 con 0x7fd68813fe40 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 >> v1:172.21.15.70:6790/0 conn(0x7fd6880b0570 legacy=0x7fd68813b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 >> v1:172.21.15.172:6789/0 conn(0x7fd688139390 legacy=0x7fd6881268d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68813a880 con 0x7fd68813fe40 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd68cb07640 1 -- 172.21.15.70:0/3550199994 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd688129440 con 0x7fd68813fe40 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd68cb07640 1 -- 172.21.15.70:0/3550199994 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd68813a120 con 0x7fd68813fe40 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd680001b90 con 0x7fd68813fe40 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.671+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd680003e80 con 0x7fd68813fe40 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.672+0000 7fd68cb07640 1 -- 172.21.15.70:0/3550199994 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66c005080 con 0x7fd68813fe40 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.673+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fd68001cc50 con 0x7fd68813fe40 2024-09-18T07:14:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.673+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7fd6800950f0 con 0x7fd68813fe40 2024-09-18T07:14:15.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.678+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7fd68813b970 con 0x7fd68813fe40 2024-09-18T07:14:15.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.897+0000 7fd68cb07640 1 -- 172.21.15.70:0/3550199994 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd66c005370 con 0x7fd68813fe40 2024-09-18T07:14:15.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.899+0000 7fd67effd640 1 -- 172.21.15.70:0/3550199994 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194654 (unknown 481731318 0 4128145197) 0x7fd68005dac0 con 0x7fd68813fe40 2024-09-18T07:14:15.905 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:14:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.902+0000 7fd68cb07640 1 -- 172.21.15.70:0/3550199994 >> v1:172.21.15.70:6816/1657990791 conn(0x7fd6680798d0 legacy=0x7fd66807bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.902+0000 7fd68cb07640 1 -- 172.21.15.70:0/3550199994 >> v1:172.21.15.70:6789/0 conn(0x7fd68813fe40 legacy=0x7fd688126fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:15.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.902+0000 7fd68cb07640 1 -- 172.21.15.70:0/3550199994 shutdown_connections 2024-09-18T07:14:15.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.902+0000 7fd68cb07640 1 -- 172.21.15.70:0/3550199994 >> 172.21.15.70:0/3550199994 conn(0x7fd6880b1b70 msgr2=0x7fd6880afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:15.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.902+0000 7fd68cb07640 1 -- 172.21.15.70:0/3550199994 shutdown_connections 2024-09-18T07:14:15.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:15.902+0000 7fd68cb07640 1 -- 172.21.15.70:0/3550199994 wait complete. 2024-09-18T07:14:15.915 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6816","nonce":1657990791}]},"active_addr":"172.21.15.70:6816/1657990791","active_change":"2024-09-18T07:13:57.439302+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4107,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3206404275}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":275470310}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3220521142}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1117431352}]}]} 2024-09-18T07:14:15.919 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-18T07:14:15.919 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T07:14:15.919 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T07:14:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.015+0000 7fa72975a640 1 Processor -- start 2024-09-18T07:14:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.015+0000 7fa72975a640 1 -- start start 2024-09-18T07:14:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa72975a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7240ac2b0 con 0x7fa7241467b0 2024-09-18T07:14:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa72975a640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7240b55b0 con 0x7fa7240aa950 2024-09-18T07:14:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa72975a640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa724146c40 con 0x7fa7241404f0 2024-09-18T07:14:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa7227fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa7241467b0 0x7fa7240b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51732/0 (socket says 172.21.15.70:51732) 2024-09-18T07:14:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa7227fc640 1 -- 172.21.15.70:0/2526318232 learned_addr learned my addr 172.21.15.70:0/2526318232 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa7237fe640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fa7241404f0 0x7fa72414eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37544/0 (socket says 172.21.15.70:37544) 2024-09-18T07:14:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa721ffb640 1 -- 172.21.15.70:0/2526318232 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1680153863 0 0) 0x7fa7240ac2b0 con 0x7fa7241467b0 2024-09-18T07:14:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa721ffb640 1 -- 172.21.15.70:0/2526318232 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa70c003520 con 0x7fa7241467b0 2024-09-18T07:14:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa721ffb640 1 -- 172.21.15.70:0/2526318232 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4226800722 0 0) 0x7fa724146c40 con 0x7fa7241404f0 2024-09-18T07:14:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa721ffb640 1 -- 172.21.15.70:0/2526318232 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7240ac2b0 con 0x7fa7241404f0 2024-09-18T07:14:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa721ffb640 1 -- 172.21.15.70:0/2526318232 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1716685674 0 0) 0x7fa7240b55b0 con 0x7fa7240aa950 2024-09-18T07:14:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa721ffb640 1 -- 172.21.15.70:0/2526318232 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa724146c40 con 0x7fa7240aa950 2024-09-18T07:14:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa721ffb640 1 -- 172.21.15.70:0/2526318232 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2543403850 0 0) 0x7fa70c003520 con 0x7fa7241467b0 2024-09-18T07:14:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa721ffb640 1 -- 172.21.15.70:0/2526318232 >> v1:172.21.15.70:6790/0 conn(0x7fa7241404f0 legacy=0x7fa72414eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa721ffb640 1 -- 172.21.15.70:0/2526318232 >> v1:172.21.15.172:6789/0 conn(0x7fa7240aa950 legacy=0x7fa724147160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa721ffb640 1 -- 172.21.15.70:0/2526318232 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7240ab3f0 con 0x7fa7241467b0 2024-09-18T07:14:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.016+0000 7fa721ffb640 1 -- 172.21.15.70:0/2526318232 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fa718001690 con 0x7fa7241467b0 2024-09-18T07:14:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.017+0000 7fa721ffb640 1 -- 172.21.15.70:0/2526318232 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa718001830 con 0x7fa7241467b0 2024-09-18T07:14:16.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.017+0000 7fa721ffb640 1 -- 172.21.15.70:0/2526318232 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fa718001cc0 con 0x7fa7241467b0 2024-09-18T07:14:16.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.017+0000 7fa72975a640 1 -- 172.21.15.70:0/2526318232 >> v1:172.21.15.70:6789/0 conn(0x7fa7241467b0 legacy=0x7fa7240b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.017+0000 7fa72975a640 1 -- 172.21.15.70:0/2526318232 shutdown_connections 2024-09-18T07:14:16.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.017+0000 7fa72975a640 1 -- 172.21.15.70:0/2526318232 >> 172.21.15.70:0/2526318232 conn(0x7fa7240b5b30 msgr2=0x7fa7240b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.017+0000 7fa72975a640 1 -- 172.21.15.70:0/2526318232 shutdown_connections 2024-09-18T07:14:16.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.017+0000 7fa72975a640 1 -- 172.21.15.70:0/2526318232 wait complete. 2024-09-18T07:14:16.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.018+0000 7fa72975a640 1 Processor -- start 2024-09-18T07:14:16.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.018+0000 7fa72975a640 1 -- start start 2024-09-18T07:14:16.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa72975a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa724139960 con 0x7fa7241404f0 2024-09-18T07:14:16.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa72975a640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7241d6020 con 0x7fa724138370 2024-09-18T07:14:16.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa72975a640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa724138830 con 0x7fa7240aa950 2024-09-18T07:14:16.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa722ffd640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fa7240aa950 0x7fa7241dbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37556/0 (socket says 172.21.15.70:37556) 2024-09-18T07:14:16.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa722ffd640 1 -- 172.21.15.70:0/790938518 learned_addr learned my addr 172.21.15.70:0/790938518 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa7227fc640 1 --1- 172.21.15.70:0/790938518 >> v1:172.21.15.70:6789/0 conn(0x7fa7241404f0 0x7fa7241dc3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51748/0 (socket says 172.21.15.70:51748) 2024-09-18T07:14:16.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2140564372 0 0) 0x7fa724139960 con 0x7fa7241404f0 2024-09-18T07:14:16.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa710003520 con 0x7fa7241404f0 2024-09-18T07:14:16.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3383114650 0 0) 0x7fa724138830 con 0x7fa7240aa950 2024-09-18T07:14:16.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa724139960 con 0x7fa7240aa950 2024-09-18T07:14:16.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1927326617 0 0) 0x7fa710003520 con 0x7fa7241404f0 2024-09-18T07:14:16.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa724138830 con 0x7fa7241404f0 2024-09-18T07:14:16.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fa718001c40 con 0x7fa7241404f0 2024-09-18T07:14:16.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 209791323 0 0) 0x7fa724139960 con 0x7fa7240aa950 2024-09-18T07:14:16.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa710003520 con 0x7fa7240aa950 2024-09-18T07:14:16.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fa714001cf0 con 0x7fa7240aa950 2024-09-18T07:14:16.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1606990289 0 0) 0x7fa7241d6020 con 0x7fa724138370 2024-09-18T07:14:16.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa724139960 con 0x7fa724138370 2024-09-18T07:14:16.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 836223750 0 0) 0x7fa724138830 con 0x7fa7241404f0 2024-09-18T07:14:16.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 >> v1:172.21.15.70:6790/0 conn(0x7fa7240aa950 legacy=0x7fa7241dbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 >> v1:172.21.15.172:6789/0 conn(0x7fa724138370 legacy=0x7fa7241e2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa724138a00 con 0x7fa7241404f0 2024-09-18T07:14:16.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.019+0000 7fa72975a640 1 -- 172.21.15.70:0/790938518 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa7241e44f0 con 0x7fa7241404f0 2024-09-18T07:14:16.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.020+0000 7fa72975a640 1 -- 172.21.15.70:0/790938518 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa7241e48a0 con 0x7fa7241404f0 2024-09-18T07:14:16.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.020+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa718004a90 con 0x7fa7241404f0 2024-09-18T07:14:16.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.020+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fa718004de0 con 0x7fa7241404f0 2024-09-18T07:14:16.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.021+0000 7fa72975a640 1 -- 172.21.15.70:0/790938518 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa704005080 con 0x7fa7241404f0 2024-09-18T07:14:16.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.022+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fa71801dbb0 con 0x7fa7241404f0 2024-09-18T07:14:16.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.022+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7fa718096030 con 0x7fa7241404f0 2024-09-18T07:14:16.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.027+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7fa71805e9a0 con 0x7fa7241404f0 2024-09-18T07:14:16.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.201+0000 7fa72975a640 1 -- 172.21.15.70:0/790938518 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa704005370 con 0x7fa7241404f0 2024-09-18T07:14:16.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.202+0000 7fa70b7fe640 1 -- 172.21.15.70:0/790938518 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+12919 (unknown 898900276 0 1022960409) 0x7fa71805e2a0 con 0x7fa7241404f0 2024-09-18T07:14:16.204 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:14:16.205 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":21,"fsid":"1fa528ad-1630-4e95-9001-978a53f2a42c","created":"2024-09-18T07:13:53.234326+0000","modified":"2024-09-18T07:14:15.246247+0000","last_up_change":"2024-09-18T07:14:04.459974+0000","last_in_change":"2024-09-18T07:13:55.403239+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T07:14:05.461421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T07:14:11.323426+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4a2c5fa-37b8-4776-bf81-f142248da0da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":1022808125}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":1022808125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":1022808125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":1022808125}]},"public_addr":"172.21.15.70:6808/1022808125","cluster_addr":"172.21.15.70:6809/1022808125","heartbeat_back_addr":"172.21.15.70:6811/1022808125","heartbeat_front_addr":"172.21.15.70:6810/1022808125","state":["exists","up"]},{"osd":1,"uuid":"3fab2225-9881-4caf-8142-41b857b9b99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":1678886977}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":1678886977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":1678886977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":1678886977}]},"public_addr":"172.21.15.70:6800/1678886977","cluster_addr":"172.21.15.70:6801/1678886977","heartbeat_back_addr":"172.21.15.70:6803/1678886977","heartbeat_front_addr":"172.21.15.70:6802/1678886977","state":["exists","up"]},{"osd":2,"uuid":"5d6764cd-e7a4-45df-9cfb-88d9e9a2675f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":3042930971}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":3042930971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":3042930971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":3042930971}]},"public_addr":"172.21.15.70:6812/3042930971","cluster_addr":"172.21.15.70:6813/3042930971","heartbeat_back_addr":"172.21.15.70:6815/3042930971","heartbeat_front_addr":"172.21.15.70:6814/3042930971","state":["exists","up"]},{"osd":3,"uuid":"80961d78-b7ae-416d-bb6f-a2ef724c6cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":202753903}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":202753903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":202753903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":202753903}]},"public_addr":"172.21.15.70:6804/202753903","cluster_addr":"172.21.15.70:6805/202753903","heartbeat_back_addr":"172.21.15.70:6807/202753903","heartbeat_front_addr":"172.21.15.70:6806/202753903","state":["exists","up"]},{"osd":4,"uuid":"ac72b675-4df6-4668-8461-3d0d8ac698b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6808","nonce":3405680289}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6809","nonce":3405680289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6811","nonce":3405680289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6810","nonce":3405680289}]},"public_addr":"172.21.15.172:6808/3405680289","cluster_addr":"172.21.15.172:6809/3405680289","heartbeat_back_addr":"172.21.15.172:6811/3405680289","heartbeat_front_addr":"172.21.15.172:6810/3405680289","state":["exists","up"]},{"osd":5,"uuid":"5949328d-408f-4b72-8e51-db0046fbb090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6804","nonce":3053548626}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6805","nonce":3053548626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6807","nonce":3053548626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6806","nonce":3053548626}]},"public_addr":"172.21.15.172:6804/3053548626","cluster_addr":"172.21.15.172:6805/3053548626","heartbeat_back_addr":"172.21.15.172:6807/3053548626","heartbeat_front_addr":"172.21.15.172:6806/3053548626","state":["exists","up"]},{"osd":6,"uuid":"84ce0e0b-9072-4ef7-9e55-f3d81283b3de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6812","nonce":371905845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6813","nonce":371905845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6815","nonce":371905845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6814","nonce":371905845}]},"public_addr":"172.21.15.172:6812/371905845","cluster_addr":"172.21.15.172:6813/371905845","heartbeat_back_addr":"172.21.15.172:6815/371905845","heartbeat_front_addr":"172.21.15.172:6814/371905845","state":["exists","up"]},{"osd":7,"uuid":"96889def-b533-4738-bb6d-dd8eb99e1e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6800","nonce":2423240725}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6801","nonce":2423240725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6803","nonce":2423240725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6802","nonce":2423240725}]},"public_addr":"172.21.15.172:6800/2423240725","cluster_addr":"172.21.15.172:6801/2423240725","heartbeat_back_addr":"172.21.15.172:6803/2423240725","heartbeat_front_addr":"172.21.15.172:6802/2423240725","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.866556+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.704725+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:02.409291+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.477089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:02.747153+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.980019+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.715388+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.500717+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T07:14:16.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.204+0000 7fa72975a640 1 -- 172.21.15.70:0/790938518 >> v1:172.21.15.70:6816/1657990791 conn(0x7fa710079880 legacy=0x7fa71007bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.204+0000 7fa72975a640 1 -- 172.21.15.70:0/790938518 >> v1:172.21.15.70:6789/0 conn(0x7fa7241404f0 legacy=0x7fa7241dc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.204+0000 7fa72975a640 1 -- 172.21.15.70:0/790938518 shutdown_connections 2024-09-18T07:14:16.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.204+0000 7fa72975a640 1 -- 172.21.15.70:0/790938518 >> 172.21.15.70:0/790938518 conn(0x7fa7240b5b30 msgr2=0x7fa724076130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.204+0000 7fa72975a640 1 -- 172.21.15.70:0/790938518 shutdown_connections 2024-09-18T07:14:16.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.204+0000 7fa72975a640 1 -- 172.21.15.70:0/790938518 wait complete. 2024-09-18T07:14:16.217 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T07:14:16.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T07:14:16.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.312+0000 7fe9c44ed640 1 Processor -- start 2024-09-18T07:14:16.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.312+0000 7fe9c44ed640 1 -- start start 2024-09-18T07:14:16.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c44ed640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9bc0b1410 con 0x7fe9bc149910 2024-09-18T07:14:16.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c44ed640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9bc0acaa0 con 0x7fe9bc0ae7f0 2024-09-18T07:14:16.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c44ed640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9bc13d8c0 con 0x7fe9bc0afce0 2024-09-18T07:14:16.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c2289640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fe9bc0afce0 0x7fe9bc13d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37572/0 (socket says 172.21.15.70:37572) 2024-09-18T07:14:16.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c2289640 1 -- 172.21.15.70:0/3784202167 learned_addr learned my addr 172.21.15.70:0/3784202167 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c2a8a640 1 --1- 172.21.15.70:0/3784202167 >> v1:172.21.15.70:6789/0 conn(0x7fe9bc149910 0x7fe9bc14bd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51752/0 (socket says 172.21.15.70:51752) 2024-09-18T07:14:16.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c1287640 1 -- 172.21.15.70:0/3784202167 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4242612700 0 0) 0x7fe9bc0b1410 con 0x7fe9bc149910 2024-09-18T07:14:16.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c1287640 1 -- 172.21.15.70:0/3784202167 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9a8003520 con 0x7fe9bc149910 2024-09-18T07:14:16.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c1287640 1 -- 172.21.15.70:0/3784202167 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 165922527 0 0) 0x7fe9bc13d8c0 con 0x7fe9bc0afce0 2024-09-18T07:14:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c1287640 1 -- 172.21.15.70:0/3784202167 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9bc0b1410 con 0x7fe9bc0afce0 2024-09-18T07:14:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c1287640 1 -- 172.21.15.70:0/3784202167 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2531918500 0 0) 0x7fe9a8003520 con 0x7fe9bc149910 2024-09-18T07:14:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c1287640 1 -- 172.21.15.70:0/3784202167 >> v1:172.21.15.70:6790/0 conn(0x7fe9bc0afce0 legacy=0x7fe9bc13d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c1287640 1 -- 172.21.15.70:0/3784202167 >> v1:172.21.15.172:6789/0 conn(0x7fe9bc0ae7f0 legacy=0x7fe9bc0abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c1287640 1 -- 172.21.15.70:0/3784202167 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9bc074b40 con 0x7fe9bc149910 2024-09-18T07:14:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.313+0000 7fe9c1287640 1 -- 172.21.15.70:0/3784202167 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fe9b4001a10 con 0x7fe9bc149910 2024-09-18T07:14:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.314+0000 7fe9c1287640 1 -- 172.21.15.70:0/3784202167 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe9b4001c30 con 0x7fe9bc149910 2024-09-18T07:14:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.314+0000 7fe9c1287640 1 -- 172.21.15.70:0/3784202167 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fe9b4002040 con 0x7fe9bc149910 2024-09-18T07:14:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.314+0000 7fe9c44ed640 1 -- 172.21.15.70:0/3784202167 >> v1:172.21.15.70:6789/0 conn(0x7fe9bc149910 legacy=0x7fe9bc14bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.314+0000 7fe9c44ed640 1 -- 172.21.15.70:0/3784202167 shutdown_connections 2024-09-18T07:14:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.314+0000 7fe9c44ed640 1 -- 172.21.15.70:0/3784202167 >> 172.21.15.70:0/3784202167 conn(0x7fe9bc0b1b70 msgr2=0x7fe9bc0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.314+0000 7fe9c44ed640 1 -- 172.21.15.70:0/3784202167 shutdown_connections 2024-09-18T07:14:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.314+0000 7fe9c44ed640 1 -- 172.21.15.70:0/3784202167 wait complete. 2024-09-18T07:14:16.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.315+0000 7fe9c44ed640 1 Processor -- start 2024-09-18T07:14:16.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.315+0000 7fe9c44ed640 1 -- start start 2024-09-18T07:14:16.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9c44ed640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9bc141c70 con 0x7fe9bc149910 2024-09-18T07:14:16.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9c44ed640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9bc13fdd0 con 0x7fe9bc0ae7f0 2024-09-18T07:14:16.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9c44ed640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9bc1d5780 con 0x7fe9bc0afce0 2024-09-18T07:14:16.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9c1a88640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fe9bc0afce0 0x7fe9bc13f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37576/0 (socket says 172.21.15.70:37576) 2024-09-18T07:14:16.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9c1a88640 1 -- 172.21.15.70:0/672373055 learned_addr learned my addr 172.21.15.70:0/672373055 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388560783 0 0) 0x7fe9bc1d5780 con 0x7fe9bc0afce0 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9a0003520 con 0x7fe9bc0afce0 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2298070609 0 0) 0x7fe9bc141c70 con 0x7fe9bc149910 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9bc1d5780 con 0x7fe9bc149910 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2574878644 0 0) 0x7fe9bc13fdd0 con 0x7fe9bc0ae7f0 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9bc141c70 con 0x7fe9bc0ae7f0 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3201882082 0 0) 0x7fe9a0003520 con 0x7fe9bc0afce0 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe9bc13fdd0 con 0x7fe9bc0afce0 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fe9ac001990 con 0x7fe9bc0afce0 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2980700715 0 0) 0x7fe9bc1d5780 con 0x7fe9bc149910 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe9a0003520 con 0x7fe9bc149910 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fe9b4001f80 con 0x7fe9bc149910 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2502238655 0 0) 0x7fe9bc13fdd0 con 0x7fe9bc0afce0 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 >> v1:172.21.15.172:6789/0 conn(0x7fe9bc0ae7f0 legacy=0x7fe9bc1d5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 >> v1:172.21.15.70:6789/0 conn(0x7fe9bc149910 legacy=0x7fe9bc133df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9bc13fad0 con 0x7fe9bc0afce0 2024-09-18T07:14:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.316+0000 7fe9c44ed640 1 -- 172.21.15.70:0/672373055 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe9bc1d2bb0 con 0x7fe9bc0afce0 2024-09-18T07:14:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.317+0000 7fe9c44ed640 1 -- 172.21.15.70:0/672373055 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fe9bc134500 con 0x7fe9bc0afce0 2024-09-18T07:14:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.317+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe9ac003610 con 0x7fe9bc0afce0 2024-09-18T07:14:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.317+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fe9ac0039e0 con 0x7fe9bc0afce0 2024-09-18T07:14:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.318+0000 7fe9c44ed640 1 -- 172.21.15.70:0/672373055 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a4005080 con 0x7fe9bc0afce0 2024-09-18T07:14:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.318+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fe9ac003c00 con 0x7fe9bc0afce0 2024-09-18T07:14:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.319+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7fe9ac095f80 con 0x7fe9bc0afce0 2024-09-18T07:14:16.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.323+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7fe9ac05ea30 con 0x7fe9bc0afce0 2024-09-18T07:14:16.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.498+0000 7fe9c44ed640 1 -- 172.21.15.70:0/672373055 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9a4005370 con 0x7fe9bc0afce0 2024-09-18T07:14:16.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.498+0000 7fe9b2ffd640 1 -- 172.21.15.70:0/672373055 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+12919 (unknown 898900276 0 1022960409) 0x7fe9ac05e330 con 0x7fe9bc0afce0 2024-09-18T07:14:16.501 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:14:16.501 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":21,"fsid":"1fa528ad-1630-4e95-9001-978a53f2a42c","created":"2024-09-18T07:13:53.234326+0000","modified":"2024-09-18T07:14:15.246247+0000","last_up_change":"2024-09-18T07:14:04.459974+0000","last_in_change":"2024-09-18T07:13:55.403239+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T07:14:05.461421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T07:14:11.323426+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4a2c5fa-37b8-4776-bf81-f142248da0da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":1022808125}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":1022808125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":1022808125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":1022808125}]},"public_addr":"172.21.15.70:6808/1022808125","cluster_addr":"172.21.15.70:6809/1022808125","heartbeat_back_addr":"172.21.15.70:6811/1022808125","heartbeat_front_addr":"172.21.15.70:6810/1022808125","state":["exists","up"]},{"osd":1,"uuid":"3fab2225-9881-4caf-8142-41b857b9b99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":1678886977}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":1678886977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":1678886977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":1678886977}]},"public_addr":"172.21.15.70:6800/1678886977","cluster_addr":"172.21.15.70:6801/1678886977","heartbeat_back_addr":"172.21.15.70:6803/1678886977","heartbeat_front_addr":"172.21.15.70:6802/1678886977","state":["exists","up"]},{"osd":2,"uuid":"5d6764cd-e7a4-45df-9cfb-88d9e9a2675f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":3042930971}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":3042930971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":3042930971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":3042930971}]},"public_addr":"172.21.15.70:6812/3042930971","cluster_addr":"172.21.15.70:6813/3042930971","heartbeat_back_addr":"172.21.15.70:6815/3042930971","heartbeat_front_addr":"172.21.15.70:6814/3042930971","state":["exists","up"]},{"osd":3,"uuid":"80961d78-b7ae-416d-bb6f-a2ef724c6cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":202753903}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":202753903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":202753903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":202753903}]},"public_addr":"172.21.15.70:6804/202753903","cluster_addr":"172.21.15.70:6805/202753903","heartbeat_back_addr":"172.21.15.70:6807/202753903","heartbeat_front_addr":"172.21.15.70:6806/202753903","state":["exists","up"]},{"osd":4,"uuid":"ac72b675-4df6-4668-8461-3d0d8ac698b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6808","nonce":3405680289}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6809","nonce":3405680289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6811","nonce":3405680289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6810","nonce":3405680289}]},"public_addr":"172.21.15.172:6808/3405680289","cluster_addr":"172.21.15.172:6809/3405680289","heartbeat_back_addr":"172.21.15.172:6811/3405680289","heartbeat_front_addr":"172.21.15.172:6810/3405680289","state":["exists","up"]},{"osd":5,"uuid":"5949328d-408f-4b72-8e51-db0046fbb090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6804","nonce":3053548626}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6805","nonce":3053548626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6807","nonce":3053548626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6806","nonce":3053548626}]},"public_addr":"172.21.15.172:6804/3053548626","cluster_addr":"172.21.15.172:6805/3053548626","heartbeat_back_addr":"172.21.15.172:6807/3053548626","heartbeat_front_addr":"172.21.15.172:6806/3053548626","state":["exists","up"]},{"osd":6,"uuid":"84ce0e0b-9072-4ef7-9e55-f3d81283b3de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6812","nonce":371905845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6813","nonce":371905845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6815","nonce":371905845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6814","nonce":371905845}]},"public_addr":"172.21.15.172:6812/371905845","cluster_addr":"172.21.15.172:6813/371905845","heartbeat_back_addr":"172.21.15.172:6815/371905845","heartbeat_front_addr":"172.21.15.172:6814/371905845","state":["exists","up"]},{"osd":7,"uuid":"96889def-b533-4738-bb6d-dd8eb99e1e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6800","nonce":2423240725}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6801","nonce":2423240725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6803","nonce":2423240725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6802","nonce":2423240725}]},"public_addr":"172.21.15.172:6800/2423240725","cluster_addr":"172.21.15.172:6801/2423240725","heartbeat_back_addr":"172.21.15.172:6803/2423240725","heartbeat_front_addr":"172.21.15.172:6802/2423240725","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.866556+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.704725+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:02.409291+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.477089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:02.747153+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.980019+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.715388+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.500717+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T07:14:16.503 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.501+0000 7fe9c44ed640 1 -- 172.21.15.70:0/672373055 >> v1:172.21.15.70:6816/1657990791 conn(0x7fe9a007ba40 legacy=0x7fe9a007df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.503 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.501+0000 7fe9c44ed640 1 -- 172.21.15.70:0/672373055 >> v1:172.21.15.70:6790/0 conn(0x7fe9bc0afce0 legacy=0x7fe9bc13f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.501+0000 7fe9c44ed640 1 -- 172.21.15.70:0/672373055 shutdown_connections 2024-09-18T07:14:16.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.501+0000 7fe9c44ed640 1 -- 172.21.15.70:0/672373055 >> 172.21.15.70:0/672373055 conn(0x7fe9bc0b1b70 msgr2=0x7fe9bc0b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.501+0000 7fe9c44ed640 1 -- 172.21.15.70:0/672373055 shutdown_connections 2024-09-18T07:14:16.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.501+0000 7fe9c44ed640 1 -- 172.21.15.70:0/672373055 wait complete. 2024-09-18T07:14:16.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T07:14:16.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T07:14:16.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T07:14:16.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T07:14:16.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T07:14:16.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T07:14:16.517 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T07:14:16.517 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T07:14:16.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.679+0000 7fcc2121f640 1 Processor -- start 2024-09-18T07:14:16.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.679+0000 7fcc2121f640 1 -- start start 2024-09-18T07:14:16.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.679+0000 7f6c997b4640 1 Processor -- start 2024-09-18T07:14:16.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.680+0000 7fcc2121f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc1c1486a0 con 0x7fcc1c150ff0 2024-09-18T07:14:16.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.680+0000 7fcc1a59c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fcc1c0b3950 0x7fcc1c144b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37590/0 (socket says 172.21.15.70:37590) 2024-09-18T07:14:16.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.680+0000 7f4465911640 1 Processor -- start 2024-09-18T07:14:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.680+0000 7fcc1a59c640 1 -- 172.21.15.70:0/3183553418 learned_addr learned my addr 172.21.15.70:0/3183553418 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7fcc2121f640 1 -- 172.21.15.70:0/3183553418 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc1c06f140 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7fcc2121f640 1 -- 172.21.15.70:0/3183553418 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc1c070290 con 0x7fcc1c0b3950 2024-09-18T07:14:16.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.680+0000 7f6c997b4640 1 -- start start 2024-09-18T07:14:16.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7fcc19d9b640 1 -- 172.21.15.70:0/3183553418 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1781756418 0 0) 0x7fcc1c06f140 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7fcc19d9b640 1 -- 172.21.15.70:0/3183553418 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc04003520 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7f6c927fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6c8c030a70 0x7f6c8c0317b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51786/0 (socket says 172.21.15.70:51786) 2024-09-18T07:14:16.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7f4465911640 1 -- start start 2024-09-18T07:14:16.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7fcc19d9b640 1 -- 172.21.15.70:0/3183553418 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1221566454 0 0) 0x7fcc1c070290 con 0x7fcc1c0b3950 2024-09-18T07:14:16.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7fcc19d9b640 1 -- 172.21.15.70:0/3183553418 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc1c06f140 con 0x7fcc1c0b3950 2024-09-18T07:14:16.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7fcc19d9b640 1 -- 172.21.15.70:0/3183553418 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1446053002 0 0) 0x7fcc04003520 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7fcc19d9b640 1 -- 172.21.15.70:0/3183553418 >> v1:172.21.15.70:6790/0 conn(0x7fcc1c0b3950 legacy=0x7fcc1c144b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7fcc19d9b640 1 -- 172.21.15.70:0/3183553418 >> v1:172.21.15.70:6789/0 conn(0x7fcc1c150ff0 legacy=0x7fcc1c1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7fcc19d9b640 1 -- 172.21.15.70:0/3183553418 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c0b35e0 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7fcc19d9b640 1 -- 172.21.15.70:0/3183553418 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fcc0c001610 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7f6c927fc640 1 -- 172.21.15.70:0/1682010696 learned_addr learned my addr 172.21.15.70:0/1682010696 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7f0b138ed640 1 Processor -- start 2024-09-18T07:14:16.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7fcc19d9b640 1 -- 172.21.15.70:0/3183553418 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcc0c001870 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.682+0000 7fcc19d9b640 1 -- 172.21.15.70:0/3183553418 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fcc0c001c40 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.681+0000 7f6c997b4640 1 -- 172.21.15.70:0/1682010696 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c8c02fa70 con 0x7f6c8c030a70 2024-09-18T07:14:16.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.682+0000 7f6c997b4640 1 -- 172.21.15.70:0/1682010696 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c8c031ec0 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.682+0000 7f6c997b4640 1 -- 172.21.15.70:0/1682010696 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c8c032cc0 con 0x7f6c8c007210 2024-09-18T07:14:16.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.682+0000 7f6743c5a640 1 Processor -- start 2024-09-18T07:14:16.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.682+0000 7f6743c5a640 1 -- start start 2024-09-18T07:14:16.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.682+0000 7f6c91ffb640 1 -- 172.21.15.70:0/1682010696 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1598245972 0 0) 0x7f6c8c02fa70 con 0x7f6c8c030a70 2024-09-18T07:14:16.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.682+0000 7f6c91ffb640 1 -- 172.21.15.70:0/1682010696 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c7c003520 con 0x7f6c8c030a70 2024-09-18T07:14:16.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.682+0000 7f6c91ffb640 1 -- 172.21.15.70:0/1682010696 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 475420103 0 0) 0x7f6c8c031ec0 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.682+0000 7f6c91ffb640 1 -- 172.21.15.70:0/1682010696 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c8c02fa70 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f4465911640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f445802fa40 con 0x7f44580071e0 2024-09-18T07:14:16.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7fcc2121f640 1 -- 172.21.15.70:0/3183553418 >> v1:172.21.15.172:6789/0 conn(0x7fcc1c0b63d0 legacy=0x7fcc1c147f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f4465911640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4458031e90 con 0x7f44580c6980 2024-09-18T07:14:16.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.682+0000 7f0b138ed640 1 -- start start 2024-09-18T07:14:16.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7fcc2121f640 1 -- 172.21.15.70:0/3183553418 shutdown_connections 2024-09-18T07:14:16.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f4465110640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f44580c6980 0x7f44580ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49528/0 (socket says 172.21.15.70:49528) 2024-09-18T07:14:16.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f446490f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f44580071e0 0x7f445802f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51790/0 (socket says 172.21.15.70:51790) 2024-09-18T07:14:16.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f446490f640 1 -- 172.21.15.70:0/2600244309 learned_addr learned my addr 172.21.15.70:0/2600244309 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f4465911640 1 -- 172.21.15.70:0/2600244309 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4458032c90 con 0x7f4458030a40 2024-09-18T07:14:16.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7fcc2121f640 1 -- 172.21.15.70:0/3183553418 >> 172.21.15.70:0/3183553418 conn(0x7fcc1c0b9580 msgr2=0x7fcc1c0b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f445f7fe640 1 -- 172.21.15.70:0/2600244309 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2478849548 0 0) 0x7f4458031e90 con 0x7f44580c6980 2024-09-18T07:14:16.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f6c91ffb640 1 -- 172.21.15.70:0/1682010696 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1779110768 0 0) 0x7f6c7c003520 con 0x7f6c8c030a70 2024-09-18T07:14:16.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f6c91ffb640 1 -- 172.21.15.70:0/1682010696 >> v1:172.21.15.70:6790/0 conn(0x7f6c8c007210 legacy=0x7f6c8c02f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f6c91ffb640 1 -- 172.21.15.70:0/1682010696 >> v1:172.21.15.172:6789/0 conn(0x7f6c8c0c69b0 legacy=0x7f6c8c0ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f6c91ffb640 1 -- 172.21.15.70:0/1682010696 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c032fa0 con 0x7f6c8c030a70 2024-09-18T07:14:16.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f6c91ffb640 1 -- 172.21.15.70:0/1682010696 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6c84001690 con 0x7f6c8c030a70 2024-09-18T07:14:16.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f445f7fe640 1 -- 172.21.15.70:0/2600244309 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4450003520 con 0x7f44580c6980 2024-09-18T07:14:16.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f6743c5a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673c0ac220 con 0x7f673c0ab0d0 2024-09-18T07:14:16.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f445f7fe640 1 -- 172.21.15.70:0/2600244309 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 306996123 0 0) 0x7f445802fa40 con 0x7f44580071e0 2024-09-18T07:14:16.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f6743c5a640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673c0ad290 con 0x7f673c0af050 2024-09-18T07:14:16.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f6c91ffb640 1 -- 172.21.15.70:0/1682010696 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6c84001830 con 0x7f6c8c030a70 2024-09-18T07:14:16.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f6c91ffb640 1 -- 172.21.15.70:0/1682010696 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6c84001c40 con 0x7f6c8c030a70 2024-09-18T07:14:16.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f445f7fe640 1 -- 172.21.15.70:0/2600244309 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4458031e90 con 0x7f44580071e0 2024-09-18T07:14:16.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7fcc2121f640 1 -- 172.21.15.70:0/3183553418 shutdown_connections 2024-09-18T07:14:16.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f6743c5a640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673c145150 con 0x7f673c0aacf0 2024-09-18T07:14:16.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f67411f5640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f673c0af050 0x7f673c1476c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49530/0 (socket says 172.21.15.70:49530) 2024-09-18T07:14:16.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7f67411f5640 1 -- 172.21.15.70:0/1038319859 learned_addr learned my addr 172.21.15.70:0/1038319859 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f0b11689640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f0b0c0715e0 0x7f0b0c140540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37614/0 (socket says 172.21.15.70:37614) 2024-09-18T07:14:16.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f0b11689640 1 -- 172.21.15.70:0/684309714 learned_addr learned my addr 172.21.15.70:0/684309714 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f0b138ed640 1 -- 172.21.15.70:0/684309714 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b0c073df0 con 0x7f0b0c0719c0 2024-09-18T07:14:16.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f0b138ed640 1 -- 172.21.15.70:0/684309714 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b0c13ffc0 con 0x7f0b0c140c50 2024-09-18T07:14:16.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7fb6797b4640 1 Processor -- start 2024-09-18T07:14:16.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f0b138ed640 1 -- 172.21.15.70:0/684309714 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b0c0b00e0 con 0x7f0b0c0715e0 2024-09-18T07:14:16.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.683+0000 7fb6797b4640 1 -- start start 2024-09-18T07:14:16.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f0b03fff640 1 -- 172.21.15.70:0/684309714 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2541081016 0 0) 0x7f0b0c0b00e0 con 0x7f0b0c0715e0 2024-09-18T07:14:16.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f0b03fff640 1 -- 172.21.15.70:0/684309714 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0af8003520 con 0x7f0b0c0715e0 2024-09-18T07:14:16.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f445f7fe640 1 -- 172.21.15.70:0/2600244309 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3289922279 0 0) 0x7f4458032c90 con 0x7f4458030a40 2024-09-18T07:14:16.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f445f7fe640 1 -- 172.21.15.70:0/2600244309 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f445802fa40 con 0x7f4458030a40 2024-09-18T07:14:16.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f445f7fe640 1 -- 172.21.15.70:0/2600244309 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 561861676 0 0) 0x7f4458031e90 con 0x7f44580071e0 2024-09-18T07:14:16.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f445f7fe640 1 -- 172.21.15.70:0/2600244309 >> v1:172.21.15.70:6790/0 conn(0x7f4458030a40 legacy=0x7f4458031780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f445f7fe640 1 -- 172.21.15.70:0/2600244309 >> v1:172.21.15.172:6789/0 conn(0x7f44580c6980 legacy=0x7f44580ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f445f7fe640 1 -- 172.21.15.70:0/2600244309 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4458032f70 con 0x7f44580071e0 2024-09-18T07:14:16.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f445f7fe640 1 -- 172.21.15.70:0/2600244309 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4454001540 con 0x7f44580071e0 2024-09-18T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7fb6797b4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb674072e60 con 0x7fb67406a180 2024-09-18T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7fb6797b4640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb674073140 con 0x7fb67413cac0 2024-09-18T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7fb6797b4640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb674071d20 con 0x7fb6740b2370 2024-09-18T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7fb672ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb67406a180 0x7fb67413c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51812/0 (socket says 172.21.15.70:51812) 2024-09-18T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7fb672ffd640 1 -- 172.21.15.70:0/2145125324 learned_addr learned my addr 172.21.15.70:0/2145125324 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f6c997b4640 1 -- 172.21.15.70:0/1682010696 >> v1:172.21.15.70:6789/0 conn(0x7f6c8c030a70 legacy=0x7f6c8c0317b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7fd5d7fff640 1 Processor -- start 2024-09-18T07:14:16.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f0b03fff640 1 -- 172.21.15.70:0/684309714 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 253750041 0 0) 0x7f0af8003520 con 0x7f0b0c0715e0 2024-09-18T07:14:16.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f0b03fff640 1 -- 172.21.15.70:0/684309714 >> v1:172.21.15.172:6789/0 conn(0x7f0b0c140c50 legacy=0x7f0b0c0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-18T07:14:16.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f0b03fff640 1 -- 172.21.15.70:0/684309714 >> v1:172.21.15.70:6789/0 conn(0x7f0b0c0719c0 legacy=0x7f0b0c14b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f0b03fff640 1 -- 172.21.15.70:0/684309714 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c0b02b0 con 0x7f0b0c0715e0 2024-09-18T07:14:16.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7f0b03fff640 1 -- 172.21.15.70:0/684309714 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f0b04001a10 con 0x7f0b0c0715e0 2024-09-18T07:14:16.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7fd5d7fff640 1 -- start start 2024-09-18T07:14:16.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f445f7fe640 1 -- 172.21.15.70:0/2600244309 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f44540016e0 con 0x7f44580071e0 2024-09-18T07:14:16.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f445f7fe640 1 -- 172.21.15.70:0/2600244309 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4454001ab0 con 0x7f44580071e0 2024-09-18T07:14:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f6c997b4640 1 -- 172.21.15.70:0/1682010696 shutdown_connections 2024-09-18T07:14:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f6c997b4640 1 -- 172.21.15.70:0/1682010696 >> 172.21.15.70:0/1682010696 conn(0x7f6c8c0060b0 msgr2=0x7f6c8c0064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f67409f4640 1 -- 172.21.15.70:0/1038319859 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3835692518 0 0) 0x7f673c0ad290 con 0x7f673c0af050 2024-09-18T07:14:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f6c997b4640 1 -- 172.21.15.70:0/1682010696 shutdown_connections 2024-09-18T07:14:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f67409f4640 1 -- 172.21.15.70:0/1038319859 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6728003520 con 0x7f673c0af050 2024-09-18T07:14:16.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f0b03fff640 1 -- 172.21.15.70:0/684309714 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0b04001ce0 con 0x7f0b0c0715e0 2024-09-18T07:14:16.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f0b03fff640 1 -- 172.21.15.70:0/684309714 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f0b040020b0 con 0x7f0b0c0715e0 2024-09-18T07:14:16.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f6c997b4640 1 -- 172.21.15.70:0/1682010696 wait complete. 2024-09-18T07:14:16.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7fb6727fc640 1 -- 172.21.15.70:0/2145125324 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 497370118 0 0) 0x7fb674072e60 con 0x7fb67406a180 2024-09-18T07:14:16.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7fb6727fc640 1 -- 172.21.15.70:0/2145125324 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb660003520 con 0x7fb67406a180 2024-09-18T07:14:16.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f67409f4640 1 -- 172.21.15.70:0/1038319859 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4265001289 0 0) 0x7f6728003520 con 0x7f673c0af050 2024-09-18T07:14:16.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f67409f4640 1 -- 172.21.15.70:0/1038319859 >> v1:172.21.15.70:6790/0 conn(0x7f673c0aacf0 legacy=0x7f673c144a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-18T07:14:16.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f67409f4640 1 -- 172.21.15.70:0/1038319859 >> v1:172.21.15.70:6789/0 conn(0x7f673c0ab0d0 legacy=0x7f673c152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f67409f4640 1 -- 172.21.15.70:0/1038319859 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c147dd0 con 0x7f673c0af050 2024-09-18T07:14:16.709 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7fb6727fc640 1 -- 172.21.15.70:0/2145125324 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2968591866 0 0) 0x7fb660003520 con 0x7fb67406a180 2024-09-18T07:14:16.709 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7fb6727fc640 1 -- 172.21.15.70:0/2145125324 >> v1:172.21.15.70:6790/0 conn(0x7fb6740b2370 legacy=0x7fb674142e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-18T07:14:16.709 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7fb6727fc640 1 -- 172.21.15.70:0/2145125324 >> v1:172.21.15.172:6789/0 conn(0x7fb67413cac0 legacy=0x7fb674142760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.709 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7fb6727fc640 1 -- 172.21.15.70:0/2145125324 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6740712f0 con 0x7fb67406a180 2024-09-18T07:14:16.709 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7fb6727fc640 1 -- 172.21.15.70:0/2145125324 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb664001670 con 0x7fb67406a180 2024-09-18T07:14:16.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7fd5d7fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d8145150 con 0x7fd5d80b3500 2024-09-18T07:14:16.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7fd5d7fff640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d80b4700 con 0x7fd5d8074180 2024-09-18T07:14:16.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7fd5d7fff640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d80b3f00 con 0x7fd5d80b3120 2024-09-18T07:14:16.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7f4465911640 1 -- 172.21.15.70:0/2600244309 >> v1:172.21.15.70:6789/0 conn(0x7f44580071e0 legacy=0x7f445802f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7f0b138ed640 1 -- 172.21.15.70:0/684309714 >> v1:172.21.15.70:6790/0 conn(0x7f0b0c0715e0 legacy=0x7f0b0c140540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7f0b138ed640 1 -- 172.21.15.70:0/684309714 shutdown_connections 2024-09-18T07:14:16.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7f0b138ed640 1 -- 172.21.15.70:0/684309714 >> 172.21.15.70:0/684309714 conn(0x7f0b0c06cd30 msgr2=0x7f0b0c06d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7f4465911640 1 -- 172.21.15.70:0/2600244309 shutdown_connections 2024-09-18T07:14:16.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7f4465911640 1 -- 172.21.15.70:0/2600244309 >> 172.21.15.70:0/2600244309 conn(0x7f4458006080 msgr2=0x7f4458006490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7f4465911640 1 -- 172.21.15.70:0/2600244309 shutdown_connections 2024-09-18T07:14:16.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7f4465911640 1 -- 172.21.15.70:0/2600244309 wait complete. 2024-09-18T07:14:16.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.684+0000 7fcc2121f640 1 -- 172.21.15.70:0/3183553418 wait complete. 2024-09-18T07:14:16.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7fcc2121f640 1 Processor -- start 2024-09-18T07:14:16.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7fb6727fc640 1 -- 172.21.15.70:0/2145125324 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb664001940 con 0x7fb67406a180 2024-09-18T07:14:16.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7fb6727fc640 1 -- 172.21.15.70:0/2145125324 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb6640014d0 con 0x7fb67406a180 2024-09-18T07:14:16.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f67409f4640 1 -- 172.21.15.70:0/1038319859 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f672c001690 con 0x7f673c0af050 2024-09-18T07:14:16.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f67409f4640 1 -- 172.21.15.70:0/1038319859 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f672c001960 con 0x7f673c0af050 2024-09-18T07:14:16.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.685+0000 7f67409f4640 1 -- 172.21.15.70:0/1038319859 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f672c001d30 con 0x7f673c0af050 2024-09-18T07:14:16.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7fd5d67fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd5d80b3500 0x7fd5d814b3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51826/0 (socket says 172.21.15.70:51826) 2024-09-18T07:14:16.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7fd5d67fc640 1 -- 172.21.15.70:0/2495944943 learned_addr learned my addr 172.21.15.70:0/2495944943 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7fd5d5ffb640 1 -- 172.21.15.70:0/2495944943 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3019262660 0 0) 0x7fd5d80b4700 con 0x7fd5d8074180 2024-09-18T07:14:16.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7fd5d5ffb640 1 -- 172.21.15.70:0/2495944943 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5c4003520 con 0x7fd5d8074180 2024-09-18T07:14:16.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7f6c997b4640 1 Processor -- start 2024-09-18T07:14:16.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7fd5d5ffb640 1 -- 172.21.15.70:0/2495944943 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3201423569 0 0) 0x7fd5d80b3f00 con 0x7fd5d80b3120 2024-09-18T07:14:16.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7fd5d5ffb640 1 -- 172.21.15.70:0/2495944943 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5d80b4700 con 0x7fd5d80b3120 2024-09-18T07:14:16.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7f6c997b4640 1 -- start start 2024-09-18T07:14:16.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7f6743c5a640 1 -- 172.21.15.70:0/1038319859 >> v1:172.21.15.172:6789/0 conn(0x7f673c0af050 legacy=0x7f673c1476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7f6743c5a640 1 -- 172.21.15.70:0/1038319859 shutdown_connections 2024-09-18T07:14:16.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7f6743c5a640 1 -- 172.21.15.70:0/1038319859 >> 172.21.15.70:0/1038319859 conn(0x7f673c0b1b70 msgr2=0x7f673c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7fd5d5ffb640 1 -- 172.21.15.70:0/2495944943 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3288813750 0 0) 0x7fd5c4003520 con 0x7fd5d8074180 2024-09-18T07:14:16.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7fd5d5ffb640 1 -- 172.21.15.70:0/2495944943 >> v1:172.21.15.70:6790/0 conn(0x7fd5d80b3120 legacy=0x7fd5d80b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7fd5d5ffb640 1 -- 172.21.15.70:0/2495944943 >> v1:172.21.15.70:6789/0 conn(0x7fd5d80b3500 legacy=0x7fd5d814b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7fd5d5ffb640 1 -- 172.21.15.70:0/2495944943 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d814aed0 con 0x7fd5d8074180 2024-09-18T07:14:16.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7fd5d5ffb640 1 -- 172.21.15.70:0/2495944943 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd5d0001690 con 0x7fd5d8074180 2024-09-18T07:14:16.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7f35315b4640 1 Processor -- start 2024-09-18T07:14:16.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7f35315b4640 1 -- start start 2024-09-18T07:14:16.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7fd5d5ffb640 1 -- 172.21.15.70:0/2495944943 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd5d00018f0 con 0x7fd5d8074180 2024-09-18T07:14:16.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7fd5d5ffb640 1 -- 172.21.15.70:0/2495944943 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd5d0001cc0 con 0x7fd5d8074180 2024-09-18T07:14:16.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7fb6797b4640 1 -- 172.21.15.70:0/2145125324 >> v1:172.21.15.70:6789/0 conn(0x7fb67406a180 legacy=0x7fb67413c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.686+0000 7fcc2121f640 1 -- start start 2024-09-18T07:14:16.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7f0b138ed640 1 -- 172.21.15.70:0/684309714 shutdown_connections 2024-09-18T07:14:16.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7f6743c5a640 1 -- 172.21.15.70:0/1038319859 shutdown_connections 2024-09-18T07:14:16.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7fb6797b4640 1 -- 172.21.15.70:0/2145125324 shutdown_connections 2024-09-18T07:14:16.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7fb6797b4640 1 -- 172.21.15.70:0/2145125324 >> 172.21.15.70:0/2145125324 conn(0x7fb6740b55b0 msgr2=0x7fb6740b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7fb6797b4640 1 -- 172.21.15.70:0/2145125324 shutdown_connections 2024-09-18T07:14:16.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7fcc1a59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fcc1c0b63d0 0x7fcc1c1e0290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49568/0 (socket says 172.21.15.70:49568) 2024-09-18T07:14:16.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7fb6797b4640 1 -- 172.21.15.70:0/2145125324 wait complete. 2024-09-18T07:14:16.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7fcc1b59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcc1c150ff0 0x7fcc1c1e80b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51840/0 (socket says 172.21.15.70:51840) 2024-09-18T07:14:16.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f4465911640 1 Processor -- start 2024-09-18T07:14:16.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f6c92ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6c8c007210 0x7f6c8c028f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49580/0 (socket says 172.21.15.70:49580) 2024-09-18T07:14:16.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f4465911640 1 -- start start 2024-09-18T07:14:16.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f6c997b4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c8c02af40 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f6c997b4640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c8c0bf670 con 0x7f6c8c007210 2024-09-18T07:14:16.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f6c937fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6c8c0c69b0 0x7f6c8c0bef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51858/0 (socket says 172.21.15.70:51858) 2024-09-18T07:14:16.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f6c997b4640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c8c0a1070 con 0x7f6c8c030a70 2024-09-18T07:14:16.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f6c937fe640 1 -- 172.21.15.70:0/2857267513 learned_addr learned my addr 172.21.15.70:0/2857267513 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7f0b138ed640 1 -- 172.21.15.70:0/684309714 wait complete. 2024-09-18T07:14:16.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7fcc2121f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc1c1dfdb0 con 0x7fcc1c150ff0 2024-09-18T07:14:16.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7fcc2121f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc1c13faa0 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7fcc2121f640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc1c1455a0 con 0x7fcc1c0b3950 2024-09-18T07:14:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7fcc1a59c640 1 -- 172.21.15.70:0/1238960784 learned_addr learned my addr 172.21.15.70:0/1238960784 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 781077478 0 0) 0x7f6c94051dd0 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c80003520 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 805200811 0 0) 0x7f6c8c0bf670 con 0x7f6c8c007210 2024-09-18T07:14:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c94051dd0 con 0x7f6c8c007210 2024-09-18T07:14:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.687+0000 7f6743c5a640 1 -- 172.21.15.70:0/1038319859 wait complete. 2024-09-18T07:14:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6743c5a640 1 Processor -- start 2024-09-18T07:14:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f0b138ed640 1 Processor -- start 2024-09-18T07:14:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4000003671 0 0) 0x7f6c80003520 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6c8c0bf670 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4285401400 0 0) 0x7f6c94051dd0 con 0x7f6c8c007210 2024-09-18T07:14:16.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6c80003520 con 0x7f6c8c007210 2024-09-18T07:14:16.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6c940772b0 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6c88001830 con 0x7f6c8c007210 2024-09-18T07:14:16.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2495944943 >> v1:172.21.15.172:6789/0 conn(0x7fd5d8074180 legacy=0x7fd5d8152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f35315b4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f352c071270 con 0x7f352c070500 2024-09-18T07:14:16.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7fb6797b4640 1 Processor -- start 2024-09-18T07:14:16.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f35315b4640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f352c0b1260 con 0x7f352c143400 2024-09-18T07:14:16.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1167150744 0 0) 0x7f6c8c0bf670 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 >> v1:172.21.15.70:6790/0 conn(0x7f6c8c030a70 legacy=0x7f6c8c0be850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 >> v1:172.21.15.172:6789/0 conn(0x7f6c8c007210 legacy=0x7f6c8c028f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f35315b4640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f352c0b08a0 con 0x7f352c0708e0 2024-09-18T07:14:16.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c02b1c0 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f352bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f352c070500 0x7f352c142cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51856/0 (socket says 172.21.15.70:51856) 2024-09-18T07:14:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f352bfff640 1 -- 172.21.15.70:0/3240252614 learned_addr learned my addr 172.21.15.70:0/3240252614 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f0b138ed640 1 -- start start 2024-09-18T07:14:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f446490f640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f44580071e0 0x7f44580c2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37672/0 (socket says 172.21.15.70:37672) 2024-09-18T07:14:16.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6c94051400 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f446490f640 1 -- 172.21.15.70:0/1810122483 learned_addr learned my addr 172.21.15.70:0/1810122483 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f352affd640 1 -- 172.21.15.70:0/3240252614 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2037791416 0 0) 0x7f352c071270 con 0x7f352c070500 2024-09-18T07:14:16.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6743c5a640 1 -- start start 2024-09-18T07:14:16.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f352affd640 1 -- 172.21.15.70:0/3240252614 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3518003520 con 0x7f352c070500 2024-09-18T07:14:16.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f352affd640 1 -- 172.21.15.70:0/3240252614 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1874461138 0 0) 0x7f352c0b08a0 con 0x7f352c0708e0 2024-09-18T07:14:16.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.688+0000 7f352affd640 1 -- 172.21.15.70:0/3240252614 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f352c071270 con 0x7f352c0708e0 2024-09-18T07:14:16.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f352affd640 1 -- 172.21.15.70:0/3240252614 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2183670097 0 0) 0x7f352c071270 con 0x7f352c0708e0 2024-09-18T07:14:16.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f352affd640 1 -- 172.21.15.70:0/3240252614 >> v1:172.21.15.172:6789/0 conn(0x7f352c143400 legacy=0x7f352c073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f352affd640 1 -- 172.21.15.70:0/3240252614 >> v1:172.21.15.70:6789/0 conn(0x7f352c070500 legacy=0x7f352c142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f352affd640 1 -- 172.21.15.70:0/3240252614 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c14eab0 con 0x7f352c0708e0 2024-09-18T07:14:16.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f352affd640 1 -- 172.21.15.70:0/3240252614 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f35140015f0 con 0x7f352c0708e0 2024-09-18T07:14:16.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 486514568 0 0) 0x7fcc1c1dfdb0 con 0x7fcc1c150ff0 2024-09-18T07:14:16.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc08003520 con 0x7fcc1c150ff0 2024-09-18T07:14:16.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7f67419f6640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f673c0ab0d0 0x7f673c0745a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37684/0 (socket says 172.21.15.70:37684) 2024-09-18T07:14:16.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f352affd640 1 -- 172.21.15.70:0/3240252614 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3514001790 con 0x7f352c0708e0 2024-09-18T07:14:16.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7f0b11689640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0b0c0715e0 0x7f0b0c0adad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51868/0 (socket says 172.21.15.70:51868) 2024-09-18T07:14:16.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7f0b11689640 1 -- 172.21.15.70:0/3042877805 learned_addr learned my addr 172.21.15.70:0/3042877805 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3300137915 0 0) 0x7fcc1c13faa0 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc1c1dfdb0 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6c94063590 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7f6743c5a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673c073910 con 0x7f673c14aec0 2024-09-18T07:14:16.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7f67421f7640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f673c14aec0 0x7f673c1f0ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51874/0 (socket says 172.21.15.70:51874) 2024-09-18T07:14:16.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7f67421f7640 1 -- 172.21.15.70:0/1719594477 learned_addr learned my addr 172.21.15.70:0/1719594477 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7f6c997b4640 1 -- 172.21.15.70:0/2857267513 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8c029690 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7f6c997b4640 1 -- 172.21.15.70:0/2857267513 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6c8c0c3520 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.689+0000 7fb6797b4640 1 -- start start 2024-09-18T07:14:16.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7f0b138ed640 1 -- 172.21.15.70:0/3042877805 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b0c0aeac0 con 0x7f0b0c0715e0 2024-09-18T07:14:16.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7f35315b4640 1 -- 172.21.15.70:0/3240252614 >> v1:172.21.15.70:6790/0 conn(0x7f352c0708e0 legacy=0x7f352c073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7f0b138ed640 1 -- 172.21.15.70:0/3042877805 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b0c0aeda0 con 0x7f0b0c140c50 2024-09-18T07:14:16.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7f0b138ed640 1 -- 172.21.15.70:0/3042877805 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b0c0af080 con 0x7f0b0c0719c0 2024-09-18T07:14:16.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3894473000 0 0) 0x7fcc1c1dfdb0 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcc1c13faa0 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fcc10001930 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7fb6797b4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb674138b30 con 0x7fb6740b2370 2024-09-18T07:14:16.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7fb6797b4640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6740aaba0 con 0x7fb67406a180 2024-09-18T07:14:16.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.690+0000 7fb6797b4640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb67413f1f0 con 0x7fb67413cac0 2024-09-18T07:14:16.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2495944943 shutdown_connections 2024-09-18T07:14:16.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2495944943 >> 172.21.15.70:0/2495944943 conn(0x7fd5d80bd1e0 msgr2=0x7fd5d80bd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f35315b4640 1 -- 172.21.15.70:0/3240252614 shutdown_connections 2024-09-18T07:14:16.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f35315b4640 1 -- 172.21.15.70:0/3240252614 >> 172.21.15.70:0/3240252614 conn(0x7f352c0b49d0 msgr2=0x7f352c0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2850724978 0 0) 0x7f0b0c0aeac0 con 0x7f0b0c0715e0 2024-09-18T07:14:16.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0af0003520 con 0x7f0b0c0715e0 2024-09-18T07:14:16.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f4465911640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44580c1f90 con 0x7f4458030a40 2024-09-18T07:14:16.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f4465911640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44580c2200 con 0x7f44580c6980 2024-09-18T07:14:16.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f4465911640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44580bfaf0 con 0x7f44580071e0 2024-09-18T07:14:16.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3932313786 0 0) 0x7f44580c1f90 con 0x7f4458030a40 2024-09-18T07:14:16.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4444003520 con 0x7f4458030a40 2024-09-18T07:14:16.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3244300851 0 0) 0x7fcc1c13faa0 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 >> v1:172.21.15.70:6790/0 conn(0x7fcc1c0b3950 legacy=0x7fcc1c139200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 >> v1:172.21.15.70:6789/0 conn(0x7fcc1c150ff0 legacy=0x7fcc1c1e80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c139910 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7fcc2121f640 1 -- 172.21.15.70:0/1238960784 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1c1e09a0 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7fcc2121f640 1 -- 172.21.15.70:0/1238960784 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fcc1c070d20 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcc100047e0 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f6c997b4640 1 -- 172.21.15.70:0/2857267513 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f6c8c038370 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7fb66a7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb6740b2370 0x7fb674137d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51878/0 (socket says 172.21.15.70:51878) 2024-09-18T07:14:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7fb66a7fc640 1 -- 172.21.15.70:0/597186221 learned_addr learned my addr 172.21.15.70:0/597186221 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3890598942 0 0) 0x7fb6740aaba0 con 0x7fb67406a180 2024-09-18T07:14:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb654003520 con 0x7fb67406a180 2024-09-18T07:14:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f6743c5a640 1 -- 172.21.15.70:0/1719594477 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673c073bf0 con 0x7f673c0af050 2024-09-18T07:14:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f6743c5a640 1 -- 172.21.15.70:0/1719594477 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673c072000 con 0x7f673c0ab0d0 2024-09-18T07:14:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4193983425 0 0) 0x7f673c072000 con 0x7f673c0ab0d0 2024-09-18T07:14:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6720003520 con 0x7f673c0ab0d0 2024-09-18T07:14:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2162605681 0 0) 0x7f6720003520 con 0x7f673c0ab0d0 2024-09-18T07:14:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f673c072000 con 0x7f673c0ab0d0 2024-09-18T07:14:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6734001c00 con 0x7f673c0ab0d0 2024-09-18T07:14:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2495944943 shutdown_connections 2024-09-18T07:14:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2495944943 wait complete. 2024-09-18T07:14:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f35315b4640 1 -- 172.21.15.70:0/3240252614 shutdown_connections 2024-09-18T07:14:16.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3266481337 0 0) 0x7f0af0003520 con 0x7f0b0c0715e0 2024-09-18T07:14:16.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0b0c0aeac0 con 0x7f0b0c0715e0 2024-09-18T07:14:16.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f0b04001990 con 0x7f0b0c0715e0 2024-09-18T07:14:16.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1732711032 0 0) 0x7f0b0c0af080 con 0x7f0b0c0719c0 2024-09-18T07:14:16.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0af0003520 con 0x7f0b0c0719c0 2024-09-18T07:14:16.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fcc10004b30 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 514047122 0 0) 0x7f4444003520 con 0x7f4458030a40 2024-09-18T07:14:16.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f44440039b0 con 0x7f4458030a40 2024-09-18T07:14:16.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.691+0000 7f35315b4640 1 -- 172.21.15.70:0/3240252614 wait complete. 2024-09-18T07:14:16.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.692+0000 7f35315b4640 1 Processor -- start 2024-09-18T07:14:16.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f444c001930 con 0x7f4458030a40 2024-09-18T07:14:16.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4118312090 0 0) 0x7f44580bfaf0 con 0x7f44580071e0 2024-09-18T07:14:16.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4444003520 con 0x7f44580071e0 2024-09-18T07:14:16.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 189375824 0 0) 0x7f4460051dd0 con 0x7f44580c6980 2024-09-18T07:14:16.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44580bfaf0 con 0x7f44580c6980 2024-09-18T07:14:16.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 567676935 0 0) 0x7f673c073bf0 con 0x7f673c0af050 2024-09-18T07:14:16.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6720003520 con 0x7f673c0af050 2024-09-18T07:14:16.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 654873378 0 0) 0x7f673c0ac220 con 0x7f673c14aec0 2024-09-18T07:14:16.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f673c073bf0 con 0x7f673c14aec0 2024-09-18T07:14:16.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2673394380 0 0) 0x7f673c072000 con 0x7f673c0ab0d0 2024-09-18T07:14:16.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 >> v1:172.21.15.172:6789/0 conn(0x7f673c0af050 legacy=0x7f673c1e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fd5d7fff640 1 Processor -- start 2024-09-18T07:14:16.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fd5d7fff640 1 -- start start 2024-09-18T07:14:16.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f6c94063730 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 >> v1:172.21.15.70:6789/0 conn(0x7f673c14aec0 legacy=0x7f673c1f0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c153fe0 con 0x7f673c0ab0d0 2024-09-18T07:14:16.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2015643814 0 0) 0x7f0b0c0aeda0 con 0x7f0b0c140c50 2024-09-18T07:14:16.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b0c0af080 con 0x7f0b0c140c50 2024-09-18T07:14:16.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1337531968 0 0) 0x7f0b0c0aeac0 con 0x7f0b0c0715e0 2024-09-18T07:14:16.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 >> v1:172.21.15.70:6790/0 conn(0x7f0b0c0719c0 legacy=0x7f0b0c0ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 >> v1:172.21.15.172:6789/0 conn(0x7f0b0c140c50 legacy=0x7f0b0c1e31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c135600 con 0x7f0b0c0715e0 2024-09-18T07:14:16.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f0b138ed640 1 -- 172.21.15.70:0/3042877805 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c0ae1e0 con 0x7f0b0c0715e0 2024-09-18T07:14:16.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f0b138ed640 1 -- 172.21.15.70:0/3042877805 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0b0c0acf00 con 0x7f0b0c0715e0 2024-09-18T07:14:16.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fcc2121f640 1 -- 172.21.15.70:0/1238960784 --> v1:172.21.15.172:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fcbfc000f80 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.694+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f6c94112670 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.694+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 --> v1:172.21.15.70:6804/202753903 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c80084340 con 0x7f6c80080690 2024-09-18T07:14:16.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 225798583 0 0) 0x7fb654003520 con 0x7fb67406a180 2024-09-18T07:14:16.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb6740aaba0 con 0x7fb67406a180 2024-09-18T07:14:16.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2926927784 0 0) 0x7fb674138b30 con 0x7fb6740b2370 2024-09-18T07:14:16.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb654003520 con 0x7fb6740b2370 2024-09-18T07:14:16.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 614135265 0 0) 0x7fb67413f1f0 con 0x7fb67413cac0 2024-09-18T07:14:16.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb674138b30 con 0x7fb67413cac0 2024-09-18T07:14:16.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb664001670 con 0x7fb67406a180 2024-09-18T07:14:16.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 138380431 0 0) 0x7fb6740aaba0 con 0x7fb67406a180 2024-09-18T07:14:16.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 >> v1:172.21.15.70:6790/0 conn(0x7fb67413cac0 legacy=0x7fb674138420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 >> v1:172.21.15.70:6789/0 conn(0x7fb6740b2370 legacy=0x7fb674137d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6741d80d0 con 0x7fb67406a180 2024-09-18T07:14:16.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f6743c5a640 1 -- 172.21.15.70:0/1719594477 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f673c074cb0 con 0x7f673c0ab0d0 2024-09-18T07:14:16.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f6743c5a640 1 -- 172.21.15.70:0/1719594477 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f673c071c50 con 0x7f673c0ab0d0 2024-09-18T07:14:16.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.694+0000 7fd5d6ffd640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fd5d8074180 0x7fd5d8070820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37722/0 (socket says 172.21.15.70:37722) 2024-09-18T07:14:16.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.694+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/2479365580 learned_addr learned my addr 172.21.15.70:0/2479365580 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.694+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d8072410 con 0x7fd5d80b3500 2024-09-18T07:14:16.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.694+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d80726f0 con 0x7fd5d80b3120 2024-09-18T07:14:16.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7f35315b4640 1 -- start start 2024-09-18T07:14:16.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.694+0000 7f352bfff640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f352c070500 0x7f352c1e4ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37720/0 (socket says 172.21.15.70:37720) 2024-09-18T07:14:16.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.694+0000 7f352bfff640 1 -- 172.21.15.70:0/226626919 learned_addr learned my addr 172.21.15.70:0/226626919 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:16.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fb6797b4640 1 -- 172.21.15.70:0/597186221 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb67413e370 con 0x7fb67406a180 2024-09-18T07:14:16.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.693+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb664004a90 con 0x7fb67406a180 2024-09-18T07:14:16.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.694+0000 7fb6797b4640 1 -- 172.21.15.70:0/597186221 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb67413f8f0 con 0x7fb67406a180 2024-09-18T07:14:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.694+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb664004cc0 con 0x7fb67406a180 2024-09-18T07:14:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7f35315b4640 1 -- 172.21.15.70:0/226626919 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f352c0ab0d0 con 0x7f352c143400 2024-09-18T07:14:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d8073400 con 0x7fd5d8074180 2024-09-18T07:14:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7f35315b4640 1 -- 172.21.15.70:0/226626919 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f352c0ab340 con 0x7f352c0708e0 2024-09-18T07:14:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fcc10004d10 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7f6743c5a640 1 -- 172.21.15.70:0/1719594477 --> v1:172.21.15.70:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f673c1e8100 con 0x7f673c0ab0d0 2024-09-18T07:14:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1631306729 0 0) 0x7fd5d8073400 con 0x7fd5d8074180 2024-09-18T07:14:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5b8003520 con 0x7fd5d8074180 2024-09-18T07:14:16.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (unknown 3512546819 0 0) 0x7f6c940d6ea0 con 0x7f6c8c0c69b0 2024-09-18T07:14:16.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 899363086 0 0) 0x7fd5d80726f0 con 0x7fd5d80b3120 2024-09-18T07:14:16.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5d8073400 con 0x7fd5d80b3120 2024-09-18T07:14:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7f35315b4640 1 -- 172.21.15.70:0/226626919 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f352c14f100 con 0x7f352c070500 2024-09-18T07:14:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3135281516 0 0) 0x7f352c0ab340 con 0x7f352c0708e0 2024-09-18T07:14:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f350c003520 con 0x7f352c0708e0 2024-09-18T07:14:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7fcc10096020 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1761665357 0 0) 0x7fd5b8003520 con 0x7fd5d8074180 2024-09-18T07:14:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd5d80726f0 con 0x7fd5d8074180 2024-09-18T07:14:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4086685572 0 0) 0x7fd5d8073400 con 0x7fd5d80b3120 2024-09-18T07:14:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd5b8003520 con 0x7fd5d80b3120 2024-09-18T07:14:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd5c8001d90 con 0x7fd5d8074180 2024-09-18T07:14:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd5c0001810 con 0x7fd5d80b3120 2024-09-18T07:14:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 --> v1:172.21.15.70:6812/3042930971 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc100047e0 con 0x7fcc08080950 2024-09-18T07:14:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== osd.3 v1:172.21.15.70:6804/202753903 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f6c80084340 con 0x7f6c80080690 2024-09-18T07:14:16.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (unknown 3512546819 0 0) 0x7fcc10096420 con 0x7fcc1c0b63d0 2024-09-18T07:14:16.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4236025901 0 0) 0x7f350c003520 con 0x7f352c0708e0 2024-09-18T07:14:16.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f352c0ab340 con 0x7f352c0708e0 2024-09-18T07:14:16.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f351c001ba0 con 0x7f352c0708e0 2024-09-18T07:14:16.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1883321941 0 0) 0x7fd5b8003520 con 0x7fd5d80b3120 2024-09-18T07:14:16.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 >> v1:172.21.15.70:6790/0 conn(0x7fd5d8074180 legacy=0x7fd5d8070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 >> v1:172.21.15.70:6789/0 conn(0x7fd5d80b3500 legacy=0x7fd5d8140680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d80730f0 con 0x7fd5d80b3120 2024-09-18T07:14:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d8070290 con 0x7fd5d80b3120 2024-09-18T07:14:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd5d8070f30 con 0x7fd5d80b3120 2024-09-18T07:14:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1812331484 0 0) 0x7f352c0ab340 con 0x7f352c0708e0 2024-09-18T07:14:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 >> v1:172.21.15.70:6790/0 conn(0x7f352c070500 legacy=0x7f352c1e4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 >> v1:172.21.15.70:6789/0 conn(0x7f352c143400 legacy=0x7f352c1f03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2607393883 0 0) 0x7f44580bfaf0 con 0x7f44580c6980 2024-09-18T07:14:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd5c0001c60 con 0x7fd5d80b3120 2024-09-18T07:14:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd5c0004980 con 0x7fd5d80b3120 2024-09-18T07:14:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4460051dd0 con 0x7f44580c6980 2024-09-18T07:14:16.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c1e2600 con 0x7f352c0708e0 2024-09-18T07:14:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f44600772b0 con 0x7f44580c6980 2024-09-18T07:14:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.696+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3509920070 0 0) 0x7f4444003520 con 0x7f44580071e0 2024-09-18T07:14:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.697+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f44580bfaf0 con 0x7f44580071e0 2024-09-18T07:14:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.697+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4454001a30 con 0x7f44580071e0 2024-09-18T07:14:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.697+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2479587227 0 0) 0x7f44440039b0 con 0x7f4458030a40 2024-09-18T07:14:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.697+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 >> v1:172.21.15.70:6790/0 conn(0x7f44580071e0 legacy=0x7f44580c2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.697+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 >> v1:172.21.15.172:6789/0 conn(0x7f44580c6980 legacy=0x7f44580bf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0b04004e10 con 0x7f0b0c0715e0 2024-09-18T07:14:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f0b04005160 con 0x7f0b0c0715e0 2024-09-18T07:14:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.695+0000 7f0af77fe640 1 -- 172.21.15.70:0/3042877805 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f0aec000f80 con 0x7f0b0c0715e0 2024-09-18T07:14:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.697+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f0b0401df30 con 0x7f0b0c0715e0 2024-09-18T07:14:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.698+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f0b04096520 con 0x7f0b0c0715e0 2024-09-18T07:14:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.698+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 --> v1:172.21.15.70:6808/1022808125 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0af0084670 con 0x7f0af00809c0 2024-09-18T07:14:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.698+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (unknown 3512546819 0 0) 0x7f0b04096920 con 0x7f0b0c0715e0 2024-09-18T07:14:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.697+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44580c3ee0 con 0x7f4458030a40 2024-09-18T07:14:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.697+0000 7f4465911640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4458028f50 con 0x7f4458030a40 2024-09-18T07:14:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.697+0000 7f4465911640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4458029290 con 0x7f4458030a40 2024-09-18T07:14:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.697+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fb66401da90 con 0x7fb67406a180 2024-09-18T07:14:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.698+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7fb6640947a0 con 0x7fb67406a180 2024-09-18T07:14:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.697+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 --> v1:172.21.15.172:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fd5d806ce50 con 0x7fd5d80b3120 2024-09-18T07:14:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.698+0000 7f444affd640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f4440000f80 con 0x7f4458030a40 2024-09-18T07:14:16.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.699+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4440000f80 con 0x7f4458030a40 2024-09-18T07:14:16.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.699+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 <== osd.2 v1:172.21.15.70:6812/3042930971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fcc100047e0 con 0x7fcc08080950 2024-09-18T07:14:16.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.699+0000 7fb6797b4640 1 -- 172.21.15.70:0/597186221 --> v1:172.21.15.172:6808/3405680289 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6740b0f20 con 0x7fb6740b0b40 2024-09-18T07:14:16.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.699+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fd5c0004b20 con 0x7fd5d80b3120 2024-09-18T07:14:16.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.699+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6734001e10 con 0x7f673c0ab0d0 2024-09-18T07:14:16.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.699+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6734003dc0 con 0x7f673c0ab0d0 2024-09-18T07:14:16.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.699+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f6734004020 con 0x7f673c0ab0d0 2024-09-18T07:14:16.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.699+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7fd5c0095ec0 con 0x7fd5d80b3120 2024-09-18T07:14:16.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.700+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 --> v1:172.21.15.172:6812/371905845 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5b8084440 con 0x7fd5b8080830 2024-09-18T07:14:16.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.700+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (unknown 3512546819 0 0) 0x7fd5c0059f40 con 0x7fd5d80b3120 2024-09-18T07:14:16.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.700+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f444c003790 con 0x7f4458030a40 2024-09-18T07:14:16.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.697+0000 7f35315b4640 1 -- 172.21.15.70:0/226626919 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f352c1526f0 con 0x7f352c0708e0 2024-09-18T07:14:16.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.697+0000 7f35315b4640 1 -- 172.21.15.70:0/226626919 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f352c151cc0 con 0x7f352c0708e0 2024-09-18T07:14:16.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.698+0000 7f35227fc640 1 -- 172.21.15.70:0/226626919 --> v1:172.21.15.172:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f352c14ba80 con 0x7f352c0708e0 2024-09-18T07:14:16.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.700+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f351c001db0 con 0x7f352c0708e0 2024-09-18T07:14:16.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.700+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f67340963c0 con 0x7f673c0ab0d0 2024-09-18T07:14:16.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.701+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 --> v1:172.21.15.172:6804/3053548626 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67200846d0 con 0x7f6720080a20 2024-09-18T07:14:16.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.701+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (unknown 3512546819 0 0) 0x7f6734096770 con 0x7f673c0ab0d0 2024-09-18T07:14:16.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.701+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f351c003c10 con 0x7f352c0708e0 2024-09-18T07:14:16.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.701+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f351c003e70 con 0x7f352c0708e0 2024-09-18T07:14:16.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.701+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 <== osd.4 v1:172.21.15.172:6808/3405680289 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fb6740b0f20 con 0x7fb6740b0b40 2024-09-18T07:14:16.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.701+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f444c003930 con 0x7f4458030a40 2024-09-18T07:14:16.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.701+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== osd.6 v1:172.21.15.172:6812/371905845 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fd5b8084440 con 0x7fd5b8080830 2024-09-18T07:14:16.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.702+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f444c094b60 con 0x7f4458030a40 2024-09-18T07:14:16.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.702+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f351c095260 con 0x7f352c0708e0 2024-09-18T07:14:16.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.702+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.70:6800/1678886977 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4444084a20 con 0x7f4444080e30 2024-09-18T07:14:16.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.702+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 <== osd.5 v1:172.21.15.172:6804/3053548626 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f67200846d0 con 0x7f6720080a20 2024-09-18T07:14:16.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.702+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (unknown 3512546819 0 0) 0x7f444c095000 con 0x7f4458030a40 2024-09-18T07:14:16.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.702+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 --> v1:172.21.15.172:6800/2423240725 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f351c0046c0 con 0x7f350c080a20 2024-09-18T07:14:16.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.702+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (unknown 3512546819 0 0) 0x7f351c095610 con 0x7f352c0708e0 2024-09-18T07:14:16.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.702+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 <== osd.0 v1:172.21.15.70:6808/1022808125 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f0af0084670 con 0x7f0af00809c0 2024-09-18T07:14:16.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.703+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 <== osd.7 v1:172.21.15.172:6800/2423240725 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f351c0046c0 con 0x7f350c080a20 2024-09-18T07:14:16.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.704+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== osd.1 v1:172.21.15.70:6800/1678886977 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f4444084a20 con 0x7f4444080e30 2024-09-18T07:14:16.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.725+0000 7f6c997b4640 1 -- 172.21.15.70:0/2857267513 --> v1:172.21.15.70:6804/202753903 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c8c02ba50 con 0x7f6c80080690 2024-09-18T07:14:16.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.725+0000 7f6c7b7fe640 1 -- 172.21.15.70:0/2857267513 <== osd.3 v1:172.21.15.70:6804/202753903 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2451364910) 0x7f6c8c02ba50 con 0x7f6c80080690 2024-09-18T07:14:16.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.726+0000 7f6c997b4640 1 -- 172.21.15.70:0/2857267513 >> v1:172.21.15.70:6804/202753903 conn(0x7f6c80080690 legacy=0x7f6c80082ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.726+0000 7f6c997b4640 1 -- 172.21.15.70:0/2857267513 >> v1:172.21.15.70:6816/1657990791 conn(0x7f6c80079570 legacy=0x7f6c8007ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.726+0000 7f6c997b4640 1 -- 172.21.15.70:0/2857267513 >> v1:172.21.15.70:6789/0 conn(0x7f6c8c0c69b0 legacy=0x7f6c8c0bef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.726+0000 7f6c937fe640 1 -- 172.21.15.70:0/2857267513 reap_dead start 2024-09-18T07:14:16.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.727+0000 7f6c997b4640 1 -- 172.21.15.70:0/2857267513 shutdown_connections 2024-09-18T07:14:16.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.727+0000 7f6c997b4640 1 -- 172.21.15.70:0/2857267513 >> 172.21.15.70:0/2857267513 conn(0x7f6c8c0060b0 msgr2=0x7f6c8c0c0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.727+0000 7f6c997b4640 1 -- 172.21.15.70:0/2857267513 shutdown_connections 2024-09-18T07:14:16.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.727+0000 7f6c997b4640 1 -- 172.21.15.70:0/2857267513 wait complete. 2024-09-18T07:14:16.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.729+0000 7fcc2121f640 1 -- 172.21.15.70:0/1238960784 --> v1:172.21.15.70:6812/3042930971 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcbfc002c70 con 0x7fcc08080950 2024-09-18T07:14:16.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.730+0000 7fcc037fe640 1 -- 172.21.15.70:0/1238960784 <== osd.2 v1:172.21.15.70:6812/3042930971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2451364910) 0x7fcbfc002c70 con 0x7fcc08080950 2024-09-18T07:14:16.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.730+0000 7fcc017fa640 1 -- 172.21.15.70:0/1238960784 >> v1:172.21.15.70:6812/3042930971 conn(0x7fcc08080950 legacy=0x7fcc08082d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.731+0000 7fcc017fa640 1 -- 172.21.15.70:0/1238960784 >> v1:172.21.15.70:6816/1657990791 conn(0x7fcc08079830 legacy=0x7fcc0807bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.731+0000 7fcc017fa640 1 -- 172.21.15.70:0/1238960784 >> v1:172.21.15.172:6789/0 conn(0x7fcc1c0b63d0 legacy=0x7fcc1c1e0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.731+0000 7fcc1b59e640 1 -- 172.21.15.70:0/1238960784 reap_dead start 2024-09-18T07:14:16.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.731+0000 7fcc017fa640 1 -- 172.21.15.70:0/1238960784 shutdown_connections 2024-09-18T07:14:16.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.731+0000 7fcc017fa640 1 -- 172.21.15.70:0/1238960784 >> 172.21.15.70:0/1238960784 conn(0x7fcc1c0b9580 msgr2=0x7fcc1c076130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.731+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 --> v1:172.21.15.172:6812/371905845 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5d8073620 con 0x7fd5b8080830 2024-09-18T07:14:16.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.732+0000 7fcc017fa640 1 -- 172.21.15.70:0/1238960784 shutdown_connections 2024-09-18T07:14:16.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.732+0000 7fcc017fa640 1 -- 172.21.15.70:0/1238960784 wait complete. 2024-09-18T07:14:16.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.732+0000 7f6743c5a640 1 -- 172.21.15.70:0/1719594477 --> v1:172.21.15.172:6804/3053548626 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f673c1e8500 con 0x7f6720080a20 2024-09-18T07:14:16.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.732+0000 7f0af77fe640 1 -- 172.21.15.70:0/3042877805 --> v1:172.21.15.70:6808/1022808125 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0aec002c70 con 0x7f0af00809c0 2024-09-18T07:14:16.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.732+0000 7fb6797b4640 1 -- 172.21.15.70:0/597186221 --> v1:172.21.15.172:6808/3405680289 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb6741407b0 con 0x7fb6740b0b40 2024-09-18T07:14:16.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.732+0000 7fd5cf7fe640 1 -- 172.21.15.70:0/2479365580 <== osd.6 v1:172.21.15.172:6812/371905845 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 700469118) 0x7fd5d0007860 con 0x7fd5b8080830 2024-09-18T07:14:16.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.732+0000 7f67327fc640 1 -- 172.21.15.70:0/1719594477 <== osd.5 v1:172.21.15.172:6804/3053548626 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 700469118) 0x7f673c1e8500 con 0x7f6720080a20 2024-09-18T07:14:16.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.732+0000 7f0b01ffb640 1 -- 172.21.15.70:0/3042877805 <== osd.0 v1:172.21.15.70:6808/1022808125 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2451364910) 0x7f0aec002c70 con 0x7f0af00809c0 2024-09-18T07:14:16.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.732+0000 7fb66bfff640 1 -- 172.21.15.70:0/597186221 <== osd.4 v1:172.21.15.172:6808/3405680289 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2451364910) 0x7fb6741407b0 con 0x7fb6740b0b40 2024-09-18T07:14:16.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.733+0000 7f6743c5a640 1 -- 172.21.15.70:0/1719594477 >> v1:172.21.15.172:6804/3053548626 conn(0x7f6720080a20 legacy=0x7f6720082e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.733+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 >> v1:172.21.15.172:6812/371905845 conn(0x7fd5b8080830 legacy=0x7fd5b8082c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.733+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 >> v1:172.21.15.70:6816/1657990791 conn(0x7fd5b8079710 legacy=0x7fd5b807bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.733+0000 7f6743c5a640 1 -- 172.21.15.70:0/1719594477 >> v1:172.21.15.70:6816/1657990791 conn(0x7f6720079990 legacy=0x7f672007be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.733+0000 7f6743c5a640 1 -- 172.21.15.70:0/1719594477 >> v1:172.21.15.70:6790/0 conn(0x7f673c0ab0d0 legacy=0x7f673c0745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.733+0000 7f0af77fe640 1 -- 172.21.15.70:0/3042877805 >> v1:172.21.15.70:6808/1022808125 conn(0x7f0af00809c0 legacy=0x7f0af0082e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.733+0000 7f67421f7640 1 -- 172.21.15.70:0/1719594477 reap_dead start 2024-09-18T07:14:16.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.733+0000 7f6743c5a640 1 -- 172.21.15.70:0/1719594477 shutdown_connections 2024-09-18T07:14:16.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.733+0000 7f6743c5a640 1 -- 172.21.15.70:0/1719594477 >> 172.21.15.70:0/1719594477 conn(0x7f673c0b1b70 msgr2=0x7f673c0ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.733+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 >> v1:172.21.15.172:6789/0 conn(0x7fd5d80b3120 legacy=0x7fd5d813ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.733+0000 7fd5d77fe640 1 -- 172.21.15.70:0/2479365580 reap_dead start 2024-09-18T07:14:16.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.733+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 shutdown_connections 2024-09-18T07:14:16.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.733+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 >> 172.21.15.70:0/2479365580 conn(0x7fd5d80bd1e0 msgr2=0x7fd5d80badb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7f0af77fe640 1 -- 172.21.15.70:0/3042877805 >> v1:172.21.15.70:6816/1657990791 conn(0x7f0af0079930 legacy=0x7f0af007bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7fb6797b4640 1 -- 172.21.15.70:0/597186221 >> v1:172.21.15.172:6808/3405680289 conn(0x7fb6740b0b40 legacy=0x7fb67413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7f0af77fe640 1 -- 172.21.15.70:0/3042877805 >> v1:172.21.15.70:6789/0 conn(0x7f0b0c0715e0 legacy=0x7f0b0c0adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7f0b11e8a640 1 -- 172.21.15.70:0/3042877805 reap_dead start 2024-09-18T07:14:16.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 shutdown_connections 2024-09-18T07:14:16.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7f0af77fe640 1 -- 172.21.15.70:0/3042877805 shutdown_connections 2024-09-18T07:14:16.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7f0af77fe640 1 -- 172.21.15.70:0/3042877805 >> 172.21.15.70:0/3042877805 conn(0x7f0b0c06cd30 msgr2=0x7f0b0c06d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7fd5d7fff640 1 -- 172.21.15.70:0/2479365580 wait complete. 2024-09-18T07:14:16.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7f0af77fe640 1 -- 172.21.15.70:0/3042877805 shutdown_connections 2024-09-18T07:14:16.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7fb6797b4640 1 -- 172.21.15.70:0/597186221 >> v1:172.21.15.70:6816/1657990791 conn(0x7fb654079cf0 legacy=0x7fb65407c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7fb6797b4640 1 -- 172.21.15.70:0/597186221 >> v1:172.21.15.172:6789/0 conn(0x7fb67406a180 legacy=0x7fb6741d79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7f6743c5a640 1 -- 172.21.15.70:0/1719594477 shutdown_connections 2024-09-18T07:14:16.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7fb6737fe640 1 -- 172.21.15.70:0/597186221 reap_dead start 2024-09-18T07:14:16.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7f35227fc640 1 -- 172.21.15.70:0/226626919 --> v1:172.21.15.172:6800/2423240725 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f352c1e55b0 con 0x7f350c080a20 2024-09-18T07:14:16.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7fb6797b4640 1 -- 172.21.15.70:0/597186221 shutdown_connections 2024-09-18T07:14:16.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7fb6797b4640 1 -- 172.21.15.70:0/597186221 >> 172.21.15.70:0/597186221 conn(0x7fb6740b55b0 msgr2=0x7fb674144210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7f0af77fe640 1 -- 172.21.15.70:0/3042877805 wait complete. 2024-09-18T07:14:16.796 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7fb6797b4640 1 -- 172.21.15.70:0/597186221 shutdown_connections 2024-09-18T07:14:16.796 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7fb6797b4640 1 -- 172.21.15.70:0/597186221 wait complete. 2024-09-18T07:14:16.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7f6743c5a640 1 -- 172.21.15.70:0/1719594477 wait complete. 2024-09-18T07:14:16.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7f3528ff9640 1 -- 172.21.15.70:0/226626919 <== osd.7 v1:172.21.15.172:6800/2423240725 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 700469118) 0x7f352c1e55b0 con 0x7f350c080a20 2024-09-18T07:14:16.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.734+0000 7f444affd640 1 -- 172.21.15.70:0/1810122483 --> v1:172.21.15.70:6800/1678886977 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4440002c70 con 0x7f4444080e30 2024-09-18T07:14:16.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.735+0000 7f445d7fa640 1 -- 172.21.15.70:0/1810122483 <== osd.1 v1:172.21.15.70:6800/1678886977 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2451364910) 0x7f4440002c70 con 0x7f4444080e30 2024-09-18T07:14:16.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.735+0000 7f35227fc640 1 -- 172.21.15.70:0/226626919 >> v1:172.21.15.172:6800/2423240725 conn(0x7f350c080a20 legacy=0x7f350c082e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.735+0000 7f35227fc640 1 -- 172.21.15.70:0/226626919 >> v1:172.21.15.70:6816/1657990791 conn(0x7f350c079900 legacy=0x7f350c07bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.735+0000 7f35227fc640 1 -- 172.21.15.70:0/226626919 >> v1:172.21.15.172:6789/0 conn(0x7f352c0708e0 legacy=0x7f352c13ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.735+0000 7f3530db3640 1 -- 172.21.15.70:0/226626919 reap_dead start 2024-09-18T07:14:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.735+0000 7f35227fc640 1 -- 172.21.15.70:0/226626919 shutdown_connections 2024-09-18T07:14:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.735+0000 7f35227fc640 1 -- 172.21.15.70:0/226626919 >> 172.21.15.70:0/226626919 conn(0x7f352c0b49d0 msgr2=0x7f352c0b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.736+0000 7f35227fc640 1 -- 172.21.15.70:0/226626919 shutdown_connections 2024-09-18T07:14:16.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.736+0000 7f444affd640 1 -- 172.21.15.70:0/1810122483 >> v1:172.21.15.70:6800/1678886977 conn(0x7f4444080e30 legacy=0x7f4444083250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.736+0000 7f35227fc640 1 -- 172.21.15.70:0/226626919 wait complete. 2024-09-18T07:14:16.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.736+0000 7f444affd640 1 -- 172.21.15.70:0/1810122483 >> v1:172.21.15.70:6816/1657990791 conn(0x7f4444079d30 legacy=0x7f444407c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.736+0000 7f444affd640 1 -- 172.21.15.70:0/1810122483 >> v1:172.21.15.70:6789/0 conn(0x7f4458030a40 legacy=0x7f44580becd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:16.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.736+0000 7f4465110640 1 -- 172.21.15.70:0/1810122483 reap_dead start 2024-09-18T07:14:16.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.736+0000 7f444affd640 1 -- 172.21.15.70:0/1810122483 shutdown_connections 2024-09-18T07:14:16.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.736+0000 7f444affd640 1 -- 172.21.15.70:0/1810122483 >> 172.21.15.70:0/1810122483 conn(0x7f4458006080 msgr2=0x7f4458037630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:16.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.736+0000 7f444affd640 1 -- 172.21.15.70:0/1810122483 shutdown_connections 2024-09-18T07:14:16.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:16.737+0000 7f444affd640 1 -- 172.21.15.70:0/1810122483 wait complete. 2024-09-18T07:14:16.806 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-18T07:14:16.806 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T07:14:16.808 INFO:teuthology.orchestra.run.smithi070.stdout:55834574852 2024-09-18T07:14:16.808 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T07:14:16.809 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-18T07:14:16.809 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T07:14:16.810 INFO:teuthology.orchestra.run.smithi070.stdout:55834574852 2024-09-18T07:14:16.810 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T07:14:16.811 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-18T07:14:16.811 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T07:14:16.812 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-18T07:14:16.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T07:14:16.813 INFO:teuthology.orchestra.run.smithi070.stdout:55834574852 2024-09-18T07:14:16.814 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T07:14:16.814 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-18T07:14:16.815 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T07:14:17.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.011+0000 7faf1f3ec640 1 Processor -- start 2024-09-18T07:14:17.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.012+0000 7faf1f3ec640 1 -- start start 2024-09-18T07:14:17.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.012+0000 7ff24a1f8640 1 Processor -- start 2024-09-18T07:14:17.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.012+0000 7ff24a1f8640 1 -- start start 2024-09-18T07:14:17.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7faf1f3ec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf10032420 con 0x7faf100caee0 2024-09-18T07:14:17.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7faf1f3ec640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf10032700 con 0x7faf10036b90 2024-09-18T07:14:17.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7faf1f3ec640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf100329e0 con 0x7faf10006b70 2024-09-18T07:14:17.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7ff24a1f8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff23c02be70 con 0x7ff23c030df0 2024-09-18T07:14:17.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7ff24a1f8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff23c0c9050 con 0x7ff23c006460 2024-09-18T07:14:17.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7ff24a1f8640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff23c0308f0 con 0x7ff23c0302b0 2024-09-18T07:14:17.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7fc13759e640 1 Processor -- start 2024-09-18T07:14:17.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7f4b9aa00640 1 Processor -- start 2024-09-18T07:14:17.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7ff351e17640 1 Processor -- start 2024-09-18T07:14:17.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7ff2489f5640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7ff23c0302b0 0x7ff23c0c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37740/0 (socket says 172.21.15.70:37740) 2024-09-18T07:14:17.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7ff2489f5640 1 -- 172.21.15.70:0/121759714 learned_addr learned my addr 172.21.15.70:0/121759714 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7faf1d989640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7faf100caee0 0x7faf100d6d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51900/0 (socket says 172.21.15.70:51900) 2024-09-18T07:14:17.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7faf1d989640 1 -- 172.21.15.70:0/323754887 learned_addr learned my addr 172.21.15.70:0/323754887 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7faf1c987640 1 --1- 172.21.15.70:0/323754887 >> v1:172.21.15.70:6790/0 conn(0x7faf10006b70 0x7faf100ca7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37730/0 (socket says 172.21.15.70:37730) 2024-09-18T07:14:17.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7ff23bfff640 1 -- 172.21.15.70:0/121759714 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2083009767 0 0) 0x7ff23c0c9050 con 0x7ff23c006460 2024-09-18T07:14:17.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7ff23bfff640 1 -- 172.21.15.70:0/121759714 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff230003520 con 0x7ff23c006460 2024-09-18T07:14:17.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7ff351e17640 1 -- start start 2024-09-18T07:14:17.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7faf0ffff640 1 -- 172.21.15.70:0/323754887 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2528035391 0 0) 0x7faf100329e0 con 0x7faf10006b70 2024-09-18T07:14:17.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7faf0ffff640 1 -- 172.21.15.70:0/323754887 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf04003520 con 0x7faf10006b70 2024-09-18T07:14:17.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7faf0ffff640 1 -- 172.21.15.70:0/323754887 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 365493699 0 0) 0x7faf10032420 con 0x7faf100caee0 2024-09-18T07:14:17.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7faf0ffff640 1 -- 172.21.15.70:0/323754887 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf100329e0 con 0x7faf100caee0 2024-09-18T07:14:17.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7ff23bfff640 1 -- 172.21.15.70:0/121759714 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2881228376 0 0) 0x7ff23c02be70 con 0x7ff23c030df0 2024-09-18T07:14:17.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7faf0ffff640 1 -- 172.21.15.70:0/323754887 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180823491 0 0) 0x7faf10032700 con 0x7faf10036b90 2024-09-18T07:14:17.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7ff23bfff640 1 -- 172.21.15.70:0/121759714 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff23c0c9050 con 0x7ff23c030df0 2024-09-18T07:14:17.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7f27c508d640 1 Processor -- start 2024-09-18T07:14:17.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7faf0ffff640 1 -- 172.21.15.70:0/323754887 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf10032420 con 0x7faf10036b90 2024-09-18T07:14:17.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.013+0000 7f4b9aa00640 1 -- start start 2024-09-18T07:14:17.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7f463aa60640 1 Processor -- start 2024-09-18T07:14:17.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7f463aa60640 1 -- start start 2024-09-18T07:14:17.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7faf0ffff640 1 -- 172.21.15.70:0/323754887 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 648933959 0 0) 0x7faf04003520 con 0x7faf10006b70 2024-09-18T07:14:17.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7faf0ffff640 1 -- 172.21.15.70:0/323754887 >> v1:172.21.15.172:6789/0 conn(0x7faf10036b90 legacy=0x7faf10032e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7faf0ffff640 1 -- 172.21.15.70:0/323754887 >> v1:172.21.15.70:6789/0 conn(0x7faf100caee0 legacy=0x7faf100d6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7faf0ffff640 1 -- 172.21.15.70:0/323754887 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf10033580 con 0x7faf10006b70 2024-09-18T07:14:17.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7faf0ffff640 1 -- 172.21.15.70:0/323754887 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7faf08001690 con 0x7faf10006b70 2024-09-18T07:14:17.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7ff23bfff640 1 -- 172.21.15.70:0/121759714 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2972399574 0 0) 0x7ff23c0308f0 con 0x7ff23c0302b0 2024-09-18T07:14:17.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7fc13759e640 1 -- start start 2024-09-18T07:14:17.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7fc13759e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc13002be70 con 0x7fc130030df0 2024-09-18T07:14:17.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7fc13759e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1300c9050 con 0x7fc130006460 2024-09-18T07:14:17.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7fc13759e640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1300308f0 con 0x7fc1300302b0 2024-09-18T07:14:17.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7ff351e17640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c073810 con 0x7ff34c074920 2024-09-18T07:14:17.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7ff351e17640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c074d70 con 0x7ff34c0714e0 2024-09-18T07:14:17.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7ff351e17640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c0720c0 con 0x7ff34c0aff60 2024-09-18T07:14:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7ff350e15640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff34c0714e0 0x7ff34c13fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49686/0 (socket says 172.21.15.70:49686) 2024-09-18T07:14:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7ff350e15640 1 -- 172.21.15.70:0/897734858 learned_addr learned my addr 172.21.15.70:0/897734858 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7faf0ffff640 1 -- 172.21.15.70:0/323754887 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7faf080018f0 con 0x7faf10006b70 2024-09-18T07:14:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7ff23bfff640 1 -- 172.21.15.70:0/121759714 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff23c02be70 con 0x7ff23c0302b0 2024-09-18T07:14:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7ff23bfff640 1 -- 172.21.15.70:0/121759714 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1031769871 0 0) 0x7ff230003520 con 0x7ff23c006460 2024-09-18T07:14:17.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7ff23bfff640 1 -- 172.21.15.70:0/121759714 >> v1:172.21.15.70:6790/0 conn(0x7ff23c0302b0 legacy=0x7ff23c0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7ff23bfff640 1 -- 172.21.15.70:0/121759714 >> v1:172.21.15.70:6789/0 conn(0x7ff23c030df0 legacy=0x7ff23c0c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7ff23bfff640 1 -- 172.21.15.70:0/121759714 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c032300 con 0x7ff23c006460 2024-09-18T07:14:17.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7ff23bfff640 1 -- 172.21.15.70:0/121759714 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff240001630 con 0x7ff23c006460 2024-09-18T07:14:17.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.014+0000 7f27c508d640 1 -- start start 2024-09-18T07:14:17.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7f27c508d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27c00abb20 con 0x7f27c00ab190 2024-09-18T07:14:17.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7f27c508d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27c00b12d0 con 0x7f27c00b0e20 2024-09-18T07:14:17.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7f27c508d640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27c0143350 con 0x7f27c00aadb0 2024-09-18T07:14:17.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f27c488c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f27c00ab190 0x7f27c00ade80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51932/0 (socket says 172.21.15.70:51932) 2024-09-18T07:14:17.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f27c488c640 1 -- 172.21.15.70:0/1779990118 learned_addr learned my addr 172.21.15.70:0/1779990118 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f463aa60640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46340ac1b0 con 0x7f4634068fd0 2024-09-18T07:14:17.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f463aa60640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46340ac490 con 0x7f46341438f0 2024-09-18T07:14:17.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f463aa60640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4634140d20 con 0x7f4634068bf0 2024-09-18T07:14:17.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f4638ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f46341438f0 0x7f463414eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49736/0 (socket says 172.21.15.70:49736) 2024-09-18T07:14:17.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f4638ffd640 1 -- 172.21.15.70:0/1508623212 learned_addr learned my addr 172.21.15.70:0/1508623212 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7ff23bfff640 1 -- 172.21.15.70:0/121759714 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff2400017d0 con 0x7ff23c006460 2024-09-18T07:14:17.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7fc136d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc130030df0 0x7fc1300c8940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51920/0 (socket says 172.21.15.70:51920) 2024-09-18T07:14:17.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7fc136d9d640 1 -- 172.21.15.70:0/3618301196 learned_addr learned my addr 172.21.15.70:0/3618301196 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7fc13559a640 1 -- 172.21.15.70:0/3618301196 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2063219406 0 0) 0x7fc13002be70 con 0x7fc130030df0 2024-09-18T07:14:17.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7fc13559a640 1 -- 172.21.15.70:0/3618301196 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc11c003520 con 0x7fc130030df0 2024-09-18T07:14:17.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7fc13559a640 1 -- 172.21.15.70:0/3618301196 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2001641204 0 0) 0x7fc1300c9050 con 0x7fc130006460 2024-09-18T07:14:17.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f27beffd640 1 -- 172.21.15.70:0/1779990118 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 404942684 0 0) 0x7f27c00abb20 con 0x7f27c00ab190 2024-09-18T07:14:17.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7fc13559a640 1 -- 172.21.15.70:0/3618301196 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc13002be70 con 0x7fc130006460 2024-09-18T07:14:17.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f27beffd640 1 -- 172.21.15.70:0/1779990118 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27a8003520 con 0x7f27c00ab190 2024-09-18T07:14:17.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f4632ffd640 1 -- 172.21.15.70:0/1508623212 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2388639653 0 0) 0x7f46340ac490 con 0x7f46341438f0 2024-09-18T07:14:17.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f4632ffd640 1 -- 172.21.15.70:0/1508623212 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4618003520 con 0x7f46341438f0 2024-09-18T07:14:17.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7faf1f3ec640 1 -- 172.21.15.70:0/323754887 >> v1:172.21.15.70:6790/0 conn(0x7faf10006b70 legacy=0x7faf100ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7ff24a1f8640 1 -- 172.21.15.70:0/121759714 >> v1:172.21.15.172:6789/0 conn(0x7ff23c006460 legacy=0x7ff23c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7f4b9aa00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b940acc60 con 0x7f4b940740c0 2024-09-18T07:14:17.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7f4b9aa00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b940ab9d0 con 0x7f4b940abc90 2024-09-18T07:14:17.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.015+0000 7f4b9aa00640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b940aad30 con 0x7f4b9414ac70 2024-09-18T07:14:17.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f4b9a1ff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4b940740c0 0x7f4b94152da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51936/0 (socket says 172.21.15.70:51936) 2024-09-18T07:14:17.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f4b9a1ff640 1 -- 172.21.15.70:0/2860483766 learned_addr learned my addr 172.21.15.70:0/2860483766 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7ff24a1f8640 1 -- 172.21.15.70:0/121759714 shutdown_connections 2024-09-18T07:14:17.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7fc13559a640 1 -- 172.21.15.70:0/3618301196 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3324396454 0 0) 0x7fc13002be70 con 0x7fc130006460 2024-09-18T07:14:17.032 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7ff34b7fe640 1 -- 172.21.15.70:0/897734858 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3886803128 0 0) 0x7ff34c073810 con 0x7ff34c074920 2024-09-18T07:14:17.032 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7ff34b7fe640 1 -- 172.21.15.70:0/897734858 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff334003520 con 0x7ff34c074920 2024-09-18T07:14:17.032 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7fc13559a640 1 -- 172.21.15.70:0/3618301196 >> v1:172.21.15.70:6790/0 conn(0x7fc1300302b0 legacy=0x7fc1300c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-18T07:14:17.032 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7fc13559a640 1 -- 172.21.15.70:0/3618301196 >> v1:172.21.15.70:6789/0 conn(0x7fc130030df0 legacy=0x7fc1300c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7fc13559a640 1 -- 172.21.15.70:0/3618301196 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc130032300 con 0x7fc130006460 2024-09-18T07:14:17.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f4632ffd640 1 -- 172.21.15.70:0/1508623212 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2262627376 0 0) 0x7f46340ac1b0 con 0x7f4634068fd0 2024-09-18T07:14:17.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f4632ffd640 1 -- 172.21.15.70:0/1508623212 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46340ac490 con 0x7f4634068fd0 2024-09-18T07:14:17.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f4632ffd640 1 -- 172.21.15.70:0/1508623212 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 673877482 0 0) 0x7f4618003520 con 0x7f46341438f0 2024-09-18T07:14:17.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f4632ffd640 1 -- 172.21.15.70:0/1508623212 >> v1:172.21.15.70:6790/0 conn(0x7f4634068bf0 legacy=0x7f463413cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-18T07:14:17.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f4632ffd640 1 -- 172.21.15.70:0/1508623212 >> v1:172.21.15.70:6789/0 conn(0x7f4634068fd0 legacy=0x7f46341431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f4632ffd640 1 -- 172.21.15.70:0/1508623212 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463413d500 con 0x7f46341438f0 2024-09-18T07:14:17.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7f4632ffd640 1 -- 172.21.15.70:0/1508623212 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f462c001990 con 0x7f46341438f0 2024-09-18T07:14:17.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7fc13559a640 1 -- 172.21.15.70:0/3618301196 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc138051400 con 0x7fc130006460 2024-09-18T07:14:17.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.016+0000 7ff24a1f8640 1 -- 172.21.15.70:0/121759714 >> 172.21.15.70:0/121759714 conn(0x7ff23c005460 msgr2=0x7ff23c005870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7f27beffd640 1 -- 172.21.15.70:0/1779990118 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 184030540 0 0) 0x7f27c00b12d0 con 0x7f27c00b0e20 2024-09-18T07:14:17.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7fc13559a640 1 -- 172.21.15.70:0/3618301196 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc138051730 con 0x7fc130006460 2024-09-18T07:14:17.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7ff34b7fe640 1 -- 172.21.15.70:0/897734858 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3727186277 0 0) 0x7ff334003520 con 0x7ff34c074920 2024-09-18T07:14:17.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7ff34b7fe640 1 -- 172.21.15.70:0/897734858 >> v1:172.21.15.70:6790/0 conn(0x7ff34c0aff60 legacy=0x7ff34c14b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7ff34b7fe640 1 -- 172.21.15.70:0/897734858 >> v1:172.21.15.172:6789/0 conn(0x7ff34c0714e0 legacy=0x7ff34c13fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7ff34b7fe640 1 -- 172.21.15.70:0/897734858 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c073a80 con 0x7ff34c074920 2024-09-18T07:14:17.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7ff34b7fe640 1 -- 172.21.15.70:0/897734858 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff340001690 con 0x7ff34c074920 2024-09-18T07:14:17.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7f4632ffd640 1 -- 172.21.15.70:0/1508623212 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f462c001b30 con 0x7f46341438f0 2024-09-18T07:14:17.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7f4632ffd640 1 -- 172.21.15.70:0/1508623212 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f462c001f80 con 0x7f46341438f0 2024-09-18T07:14:17.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7f27beffd640 1 -- 172.21.15.70:0/1779990118 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27c00abb20 con 0x7f27c00b0e20 2024-09-18T07:14:17.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7f4b989fc640 1 -- 172.21.15.70:0/2860483766 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4011914777 0 0) 0x7f4b940ab9d0 con 0x7f4b940abc90 2024-09-18T07:14:17.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7f4b989fc640 1 -- 172.21.15.70:0/2860483766 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b80003520 con 0x7f4b940abc90 2024-09-18T07:14:17.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7faf1f3ec640 1 -- 172.21.15.70:0/323754887 shutdown_connections 2024-09-18T07:14:17.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7faf1f3ec640 1 -- 172.21.15.70:0/323754887 >> 172.21.15.70:0/323754887 conn(0x7faf10039d60 msgr2=0x7faf100361b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7ff24a1f8640 1 -- 172.21.15.70:0/121759714 shutdown_connections 2024-09-18T07:14:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7f27beffd640 1 -- 172.21.15.70:0/1779990118 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1110625942 0 0) 0x7f27c0143350 con 0x7f27c00aadb0 2024-09-18T07:14:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7ff24a1f8640 1 -- 172.21.15.70:0/121759714 wait complete. 2024-09-18T07:14:17.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7f27beffd640 1 -- 172.21.15.70:0/1779990118 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27c00b12d0 con 0x7f27c00aadb0 2024-09-18T07:14:17.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7f27beffd640 1 -- 172.21.15.70:0/1779990118 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 824170826 0 0) 0x7f27c00abb20 con 0x7f27c00b0e20 2024-09-18T07:14:17.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7f27beffd640 1 -- 172.21.15.70:0/1779990118 >> v1:172.21.15.70:6790/0 conn(0x7f27c00aadb0 legacy=0x7f27c0142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7f27beffd640 1 -- 172.21.15.70:0/1779990118 >> v1:172.21.15.70:6789/0 conn(0x7f27c00ab190 legacy=0x7f27c00ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7f27beffd640 1 -- 172.21.15.70:0/1779990118 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c0143520 con 0x7f27c00b0e20 2024-09-18T07:14:17.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7f27beffd640 1 -- 172.21.15.70:0/1779990118 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f27b00019f0 con 0x7f27c00b0e20 2024-09-18T07:14:17.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7ff34b7fe640 1 -- 172.21.15.70:0/897734858 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff3400018b0 con 0x7ff34c074920 2024-09-18T07:14:17.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7ff34b7fe640 1 -- 172.21.15.70:0/897734858 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff340001cc0 con 0x7ff34c074920 2024-09-18T07:14:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7faf1f3ec640 1 -- 172.21.15.70:0/323754887 shutdown_connections 2024-09-18T07:14:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7faf1f3ec640 1 -- 172.21.15.70:0/323754887 wait complete. 2024-09-18T07:14:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7f4b989fc640 1 -- 172.21.15.70:0/2860483766 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 235823282 0 0) 0x7f4b80003520 con 0x7f4b940abc90 2024-09-18T07:14:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7f4b989fc640 1 -- 172.21.15.70:0/2860483766 >> v1:172.21.15.70:6790/0 conn(0x7f4b9414ac70 legacy=0x7f4b940af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7f4b989fc640 1 -- 172.21.15.70:0/2860483766 >> v1:172.21.15.70:6789/0 conn(0x7f4b940740c0 legacy=0x7f4b94152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7f4b989fc640 1 -- 172.21.15.70:0/2860483766 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b940b12d0 con 0x7f4b940abc90 2024-09-18T07:14:17.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7f4b989fc640 1 -- 172.21.15.70:0/2860483766 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4b84001690 con 0x7f4b940abc90 2024-09-18T07:14:17.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.017+0000 7fc13559a640 1 -- 172.21.15.70:0/3618301196 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc13806a180 con 0x7fc130006460 2024-09-18T07:14:17.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7fc13759e640 1 -- 172.21.15.70:0/3618301196 >> v1:172.21.15.172:6789/0 conn(0x7fc130006460 legacy=0x7fc1300ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7f463aa60640 1 -- 172.21.15.70:0/1508623212 >> v1:172.21.15.172:6789/0 conn(0x7f46341438f0 legacy=0x7f463414eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7f463aa60640 1 -- 172.21.15.70:0/1508623212 shutdown_connections 2024-09-18T07:14:17.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7f463aa60640 1 -- 172.21.15.70:0/1508623212 >> 172.21.15.70:0/1508623212 conn(0x7f46340b9440 msgr2=0x7f46340b9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7f27beffd640 1 -- 172.21.15.70:0/1779990118 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f27b0001b90 con 0x7f27c00b0e20 2024-09-18T07:14:17.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7f463aa60640 1 -- 172.21.15.70:0/1508623212 shutdown_connections 2024-09-18T07:14:17.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7f463aa60640 1 -- 172.21.15.70:0/1508623212 wait complete. 2024-09-18T07:14:17.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7f4b989fc640 1 -- 172.21.15.70:0/2860483766 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4b840018f0 con 0x7f4b940abc90 2024-09-18T07:14:17.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.018+0000 7ff351e17640 1 -- 172.21.15.70:0/897734858 >> v1:172.21.15.70:6789/0 conn(0x7ff34c074920 legacy=0x7ff34c0af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7ff351e17640 1 -- 172.21.15.70:0/897734858 shutdown_connections 2024-09-18T07:14:17.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7ff351e17640 1 -- 172.21.15.70:0/897734858 >> 172.21.15.70:0/897734858 conn(0x7ff34c0b5b50 msgr2=0x7ff34c0b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7ff24a1f8640 1 Processor -- start 2024-09-18T07:14:17.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7f4b989fc640 1 -- 172.21.15.70:0/2860483766 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4b84001c40 con 0x7f4b940abc90 2024-09-18T07:14:17.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7f27beffd640 1 -- 172.21.15.70:0/1779990118 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f27b0002020 con 0x7f27c00b0e20 2024-09-18T07:14:17.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7ff24a1f8640 1 -- start start 2024-09-18T07:14:17.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2860483766 >> v1:172.21.15.172:6789/0 conn(0x7f4b940abc90 legacy=0x7f4b9414b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7f27c508d640 1 -- 172.21.15.70:0/1779990118 >> v1:172.21.15.172:6789/0 conn(0x7f27c00b0e20 legacy=0x7f27c00ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7efeca368640 1 Processor -- start 2024-09-18T07:14:17.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7ff351e17640 1 -- 172.21.15.70:0/897734858 shutdown_connections 2024-09-18T07:14:17.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7ff351e17640 1 -- 172.21.15.70:0/897734858 wait complete. 2024-09-18T07:14:17.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7faf1f3ec640 1 Processor -- start 2024-09-18T07:14:17.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7fc13759e640 1 -- 172.21.15.70:0/3618301196 shutdown_connections 2024-09-18T07:14:17.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7fc13759e640 1 -- 172.21.15.70:0/3618301196 >> 172.21.15.70:0/3618301196 conn(0x7fc130005460 msgr2=0x7fc130005870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2860483766 shutdown_connections 2024-09-18T07:14:17.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2860483766 >> 172.21.15.70:0/2860483766 conn(0x7f4b940b4a40 msgr2=0x7f4b940b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7faf1f3ec640 1 -- start start 2024-09-18T07:14:17.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7f27c508d640 1 -- 172.21.15.70:0/1779990118 shutdown_connections 2024-09-18T07:14:17.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7f27c508d640 1 -- 172.21.15.70:0/1779990118 >> 172.21.15.70:0/1779990118 conn(0x7f27c00b4a40 msgr2=0x7f27c00b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7f463aa60640 1 Processor -- start 2024-09-18T07:14:17.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.020+0000 7f463aa60640 1 -- start start 2024-09-18T07:14:17.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7efeca368640 1 -- start start 2024-09-18T07:14:17.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2860483766 shutdown_connections 2024-09-18T07:14:17.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2860483766 wait complete. 2024-09-18T07:14:17.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7fc13759e640 1 -- 172.21.15.70:0/3618301196 shutdown_connections 2024-09-18T07:14:17.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.019+0000 7fc13759e640 1 -- 172.21.15.70:0/3618301196 wait complete. 2024-09-18T07:14:17.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.020+0000 7faf1d188640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7faf10006b70 0x7faf100a3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37794/0 (socket says 172.21.15.70:37794) 2024-09-18T07:14:17.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.020+0000 7ff2489f5640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7ff23c030df0 0x7ff23c1611f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37796/0 (socket says 172.21.15.70:37796) 2024-09-18T07:14:17.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.020+0000 7ff2491f6640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff23c0302b0 0x7ff23c160ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49740/0 (socket says 172.21.15.70:49740) 2024-09-18T07:14:17.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.020+0000 7ff2499f7640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff23c02ef00 0x7ff23c16b330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51960/0 (socket says 172.21.15.70:51960) 2024-09-18T07:14:17.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.020+0000 7ff24a1f8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff23c029f00 con 0x7ff23c02ef00 2024-09-18T07:14:17.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.020+0000 7f27c508d640 1 -- 172.21.15.70:0/1779990118 shutdown_connections 2024-09-18T07:14:17.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.020+0000 7ff24a1f8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff23c02a1e0 con 0x7ff23c0302b0 2024-09-18T07:14:17.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.020+0000 7ff24a1f8640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff23c02e9f0 con 0x7ff23c030df0 2024-09-18T07:14:17.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.020+0000 7ff351e17640 1 Processor -- start 2024-09-18T07:14:17.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.020+0000 7ff2489f5640 1 -- 172.21.15.70:0/320659012 learned_addr learned my addr 172.21.15.70:0/320659012 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7f27c508d640 1 -- 172.21.15.70:0/1779990118 wait complete. 2024-09-18T07:14:17.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7ff351e17640 1 -- start start 2024-09-18T07:14:17.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7f4633fff640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f4634068bf0 0x7f4634134220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37812/0 (socket says 172.21.15.70:37812) 2024-09-18T07:14:17.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7f4633fff640 1 -- 172.21.15.70:0/1900369842 learned_addr learned my addr 172.21.15.70:0/1900369842 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7f463aa60640 1 -- 172.21.15.70:0/1900369842 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46341393f0 con 0x7f46341438f0 2024-09-18T07:14:17.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7f463aa60640 1 -- 172.21.15.70:0/1900369842 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f463413f410 con 0x7f4634140510 2024-09-18T07:14:17.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7f463aa60640 1 -- 172.21.15.70:0/1900369842 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f463413f5e0 con 0x7f4634068bf0 2024-09-18T07:14:17.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7efeca368640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec4072c40 con 0x7efec413a970 2024-09-18T07:14:17.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7efeca368640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec4074680 con 0x7efec40b15d0 2024-09-18T07:14:17.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7efeca368640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec4074960 con 0x7efec4072150 2024-09-18T07:14:17.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7efec9b67640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7efec413a970 0x7efec414c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51966/0 (socket says 172.21.15.70:51966) 2024-09-18T07:14:17.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7efec9366640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7efec4072150 0x7efec4072530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37798/0 (socket says 172.21.15.70:37798) 2024-09-18T07:14:17.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7efec9366640 1 -- 172.21.15.70:0/1188659531 learned_addr learned my addr 172.21.15.70:0/1188659531 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7efebbfff640 1 -- 172.21.15.70:0/1188659531 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1519719833 0 0) 0x7efec4072c40 con 0x7efec413a970 2024-09-18T07:14:17.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7efebbfff640 1 -- 172.21.15.70:0/1188659531 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efeb0003520 con 0x7efec413a970 2024-09-18T07:14:17.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7faf1f3ec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf100a2fc0 con 0x7faf10036b90 2024-09-18T07:14:17.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7faf1f3ec640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf100a3230 con 0x7faf100caee0 2024-09-18T07:14:17.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7faf1f3ec640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf1002c420 con 0x7faf10006b70 2024-09-18T07:14:17.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7faf1c987640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7faf10036b90 0x7faf100a1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51980/0 (socket says 172.21.15.70:51980) 2024-09-18T07:14:17.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7faf1c987640 1 -- 172.21.15.70:0/897662444 learned_addr learned my addr 172.21.15.70:0/897662444 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2540918803 0 0) 0x7faf100a2fc0 con 0x7faf10036b90 2024-09-18T07:14:17.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faefc003520 con 0x7faf10036b90 2024-09-18T07:14:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091099793 0 0) 0x7ff23c02a1e0 con 0x7ff23c0302b0 2024-09-18T07:14:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff228003520 con 0x7ff23c0302b0 2024-09-18T07:14:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4211017112 0 0) 0x7ff23c029f00 con 0x7ff23c02ef00 2024-09-18T07:14:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff23c02a1e0 con 0x7ff23c02ef00 2024-09-18T07:14:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2213996154 0 0) 0x7ff23c02e9f0 con 0x7ff23c030df0 2024-09-18T07:14:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff23c029f00 con 0x7ff23c030df0 2024-09-18T07:14:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 926809171 0 0) 0x7ff228003520 con 0x7ff23c0302b0 2024-09-18T07:14:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff23c02e9f0 con 0x7ff23c0302b0 2024-09-18T07:14:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 18663421 0 0) 0x7ff23c02a1e0 con 0x7ff23c02ef00 2024-09-18T07:14:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff228003520 con 0x7ff23c02ef00 2024-09-18T07:14:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff240003790 con 0x7ff23c0302b0 2024-09-18T07:14:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff234001770 con 0x7ff23c02ef00 2024-09-18T07:14:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3846137674 0 0) 0x7ff23c029f00 con 0x7ff23c030df0 2024-09-18T07:14:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff23c02a1e0 con 0x7ff23c030df0 2024-09-18T07:14:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff244051dd0 con 0x7ff23c030df0 2024-09-18T07:14:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3713498361 0 0) 0x7ff23c02e9f0 con 0x7ff23c0302b0 2024-09-18T07:14:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 >> v1:172.21.15.70:6790/0 conn(0x7ff23c030df0 legacy=0x7ff23c1611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 >> v1:172.21.15.70:6789/0 conn(0x7ff23c02ef00 legacy=0x7ff23c16b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c15d370 con 0x7ff23c0302b0 2024-09-18T07:14:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7fc13759e640 1 Processor -- start 2024-09-18T07:14:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff24a1f8640 1 -- 172.21.15.70:0/320659012 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c028ef0 con 0x7ff23c0302b0 2024-09-18T07:14:17.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7fc13759e640 1 -- start start 2024-09-18T07:14:17.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087780652 0 0) 0x7f463413f5e0 con 0x7f4634068bf0 2024-09-18T07:14:17.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7f4b9aa00640 1 Processor -- start 2024-09-18T07:14:17.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7f4b9aa00640 1 -- start start 2024-09-18T07:14:17.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7f4b991fd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4b940abc90 0x7f4b941df9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51994/0 (socket says 172.21.15.70:51994) 2024-09-18T07:14:17.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7f4b991fd640 1 -- 172.21.15.70:0/2284101237 learned_addr learned my addr 172.21.15.70:0/2284101237 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2284101237 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b9413fd60 con 0x7f4b940abc90 2024-09-18T07:14:17.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2284101237 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b941402b0 con 0x7f4b940740c0 2024-09-18T07:14:17.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2284101237 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b940736f0 con 0x7f4b9414ac70 2024-09-18T07:14:17.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7f27c508d640 1 Processor -- start 2024-09-18T07:14:17.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7f27c508d640 1 -- start start 2024-09-18T07:14:17.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 964691881 0 0) 0x7f4b9413fd60 con 0x7f4b940abc90 2024-09-18T07:14:17.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f461c003520 con 0x7f4634068bf0 2024-09-18T07:14:17.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2381280094 0 0) 0x7f461c003520 con 0x7f4634068bf0 2024-09-18T07:14:17.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f461c0039b0 con 0x7f4634068bf0 2024-09-18T07:14:17.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7ff24a1f8640 1 -- 172.21.15.70:0/320659012 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff23c16cc30 con 0x7ff23c0302b0 2024-09-18T07:14:17.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4624003fa0 con 0x7f4634068bf0 2024-09-18T07:14:17.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.021+0000 7efebbfff640 1 -- 172.21.15.70:0/1188659531 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 482756810 0 0) 0x7efeb0003520 con 0x7efec413a970 2024-09-18T07:14:17.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7efebbfff640 1 -- 172.21.15.70:0/1188659531 >> v1:172.21.15.70:6790/0 conn(0x7efec4072150 legacy=0x7efec4072530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-18T07:14:17.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7efebbfff640 1 -- 172.21.15.70:0/1188659531 >> v1:172.21.15.172:6789/0 conn(0x7efec40b15d0 legacy=0x7efec4141450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-18T07:14:17.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1765304084 0 0) 0x7faf1002c420 con 0x7faf10006b70 2024-09-18T07:14:17.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf100a2fc0 con 0x7faf10006b70 2024-09-18T07:14:17.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2297833858 0 0) 0x7faefc003520 con 0x7faf10036b90 2024-09-18T07:14:17.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faf1002c420 con 0x7faf10036b90 2024-09-18T07:14:17.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7faf08004a20 con 0x7faf10036b90 2024-09-18T07:14:17.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7efebbfff640 1 -- 172.21.15.70:0/1188659531 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec4140f50 con 0x7efec413a970 2024-09-18T07:14:17.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7efebbfff640 1 -- 172.21.15.70:0/1188659531 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7efebc0019f0 con 0x7efec413a970 2024-09-18T07:14:17.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7efebbfff640 1 -- 172.21.15.70:0/1188659531 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efebc001cd0 con 0x7efec413a970 2024-09-18T07:14:17.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7efebbfff640 1 -- 172.21.15.70:0/1188659531 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7efebc001850 con 0x7efec413a970 2024-09-18T07:14:17.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff351e17640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c137e70 con 0x7ff34c074920 2024-09-18T07:14:17.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff351e17640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c143a30 con 0x7ff34c0aff60 2024-09-18T07:14:17.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff351e17640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c0ab560 con 0x7ff34c0714e0 2024-09-18T07:14:17.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff34bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff34c074920 0x7ff34c137050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52008/0 (socket says 172.21.15.70:52008) 2024-09-18T07:14:17.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.022+0000 7ff34bfff640 1 -- 172.21.15.70:0/720620672 learned_addr learned my addr 172.21.15.70:0/720620672 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b78003520 con 0x7f4b940abc90 2024-09-18T07:14:17.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4111987181 0 0) 0x7f463413f410 con 0x7f4634140510 2024-09-18T07:14:17.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f461c003f40 con 0x7f4634140510 2024-09-18T07:14:17.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1081767087 0 0) 0x7f46341393f0 con 0x7f46341438f0 2024-09-18T07:14:17.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3884445391 0 0) 0x7faf18051dd0 con 0x7faf100caee0 2024-09-18T07:14:17.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faefc003520 con 0x7faf100caee0 2024-09-18T07:14:17.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f463413f410 con 0x7f46341438f0 2024-09-18T07:14:17.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1898621622 0 0) 0x7f4b8c001110 con 0x7f4b940740c0 2024-09-18T07:14:17.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b9413fd60 con 0x7f4b940740c0 2024-09-18T07:14:17.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4164294865 0 0) 0x7f461c0039b0 con 0x7f4634068bf0 2024-09-18T07:14:17.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 >> v1:172.21.15.172:6789/0 conn(0x7f4634140510 legacy=0x7f46341e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 >> v1:172.21.15.70:6789/0 conn(0x7f46341438f0 legacy=0x7f46341db860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4634146f60 con 0x7f4634068bf0 2024-09-18T07:14:17.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3545875579 0 0) 0x7faf1002c420 con 0x7faf10036b90 2024-09-18T07:14:17.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 >> v1:172.21.15.70:6790/0 conn(0x7faf10006b70 legacy=0x7faf100a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 >> v1:172.21.15.172:6789/0 conn(0x7faf100caee0 legacy=0x7faf10173990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf100a4040 con 0x7faf10036b90 2024-09-18T07:14:17.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7faf1f3ec640 1 -- 172.21.15.70:0/897662444 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7faf100a1c60 con 0x7faf10036b90 2024-09-18T07:14:17.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f27c508d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27c0145ec0 con 0x7f27c00ab190 2024-09-18T07:14:17.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f463aa60640 1 -- 172.21.15.70:0/1900369842 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4634147250 con 0x7f4634068bf0 2024-09-18T07:14:17.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f27c508d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27c0143890 con 0x7f27c00b0e20 2024-09-18T07:14:17.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.023+0000 7f27c508d640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27c0144f50 con 0x7f27c00aadb0 2024-09-18T07:14:17.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7faf1f3ec640 1 -- 172.21.15.70:0/897662444 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7faf1002b0c0 con 0x7faf10036b90 2024-09-18T07:14:17.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f463aa60640 1 -- 172.21.15.70:0/1900369842 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f4634134b10 con 0x7f4634068bf0 2024-09-18T07:14:17.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f27bffff640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f27c00aadb0 0x7f27c0144840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37836/0 (socket says 172.21.15.70:37836) 2024-09-18T07:14:17.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7efeca368640 1 -- 172.21.15.70:0/1188659531 >> v1:172.21.15.70:6789/0 conn(0x7efec413a970 legacy=0x7efec414c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7fc13759e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1300cfa50 con 0x7fc130006460 2024-09-18T07:14:17.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7fc135d9b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc1300302b0 0x7fc13002d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49798/0 (socket says 172.21.15.70:49798) 2024-09-18T07:14:17.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7fc135d9b640 1 -- 172.21.15.70:0/3582376900 learned_addr learned my addr 172.21.15.70:0/3582376900 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7fc13759e640 1 -- 172.21.15.70:0/3582376900 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc130163d90 con 0x7fc1300302b0 2024-09-18T07:14:17.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7fc13759e640 1 -- 172.21.15.70:0/3582376900 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc13002efe0 con 0x7fc130030df0 2024-09-18T07:14:17.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2693238192 0 0) 0x7f4b9413fd60 con 0x7f4b940740c0 2024-09-18T07:14:17.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4b78003b40 con 0x7f4b940740c0 2024-09-18T07:14:17.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4b8c0019a0 con 0x7f4b940740c0 2024-09-18T07:14:17.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f27bffff640 1 -- 172.21.15.70:0/2809473940 learned_addr learned my addr 172.21.15.70:0/2809473940 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7efeca368640 1 -- 172.21.15.70:0/1188659531 shutdown_connections 2024-09-18T07:14:17.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7efeca368640 1 -- 172.21.15.70:0/1188659531 >> 172.21.15.70:0/1188659531 conn(0x7efec40b6bd0 msgr2=0x7efec40b6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff240001ac0 con 0x7ff23c0302b0 2024-09-18T07:14:17.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff240003d80 con 0x7ff23c0302b0 2024-09-18T07:14:17.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082153596 0 0) 0x7fc130163d90 con 0x7fc1300302b0 2024-09-18T07:14:17.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc120003520 con 0x7fc1300302b0 2024-09-18T07:14:17.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1709937107 0 0) 0x7ff34c143a30 con 0x7ff34c0aff60 2024-09-18T07:14:17.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.025+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff338003520 con 0x7ff34c0aff60 2024-09-18T07:14:17.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.025+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2786744886 0 0) 0x7ff34c0ab560 con 0x7ff34c0714e0 2024-09-18T07:14:17.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4624000ed0 con 0x7f4634068bf0 2024-09-18T07:14:17.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4624003e00 con 0x7f4634068bf0 2024-09-18T07:14:17.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.025+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3182248546 0 0) 0x7fc120003520 con 0x7fc1300302b0 2024-09-18T07:14:17.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.025+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc130163d90 con 0x7fc1300302b0 2024-09-18T07:14:17.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.025+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc128001960 con 0x7fc1300302b0 2024-09-18T07:14:17.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.025+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff34c143a30 con 0x7ff34c0714e0 2024-09-18T07:14:17.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.025+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2801872472 0 0) 0x7fc130163d90 con 0x7fc1300302b0 2024-09-18T07:14:17.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 >> v1:172.21.15.70:6790/0 conn(0x7fc130030df0 legacy=0x7fc13016c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 >> v1:172.21.15.70:6789/0 conn(0x7fc130006460 legacy=0x7fc1301642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13015d2d0 con 0x7fc1300302b0 2024-09-18T07:14:17.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2005572672 0 0) 0x7ff34c137e70 con 0x7ff34c074920 2024-09-18T07:14:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 151705739 0 0) 0x7f4b78003b40 con 0x7f4b940740c0 2024-09-18T07:14:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 >> v1:172.21.15.70:6790/0 conn(0x7f4b9414ac70 legacy=0x7f4b9411e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 >> v1:172.21.15.70:6789/0 conn(0x7f4b940abc90 legacy=0x7f4b941df9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9413ffd0 con 0x7f4b940740c0 2024-09-18T07:14:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2284101237 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4b94143ed0 con 0x7f4b940740c0 2024-09-18T07:14:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.024+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2284101237 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4b940728d0 con 0x7f4b940740c0 2024-09-18T07:14:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.025+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4b8c001c40 con 0x7f4b940740c0 2024-09-18T07:14:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff34c0ab560 con 0x7ff34c074920 2024-09-18T07:14:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2033976378 0 0) 0x7ff338003520 con 0x7ff34c0aff60 2024-09-18T07:14:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff34c137e70 con 0x7ff34c0aff60 2024-09-18T07:14:17.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7fc13759e640 1 -- 172.21.15.70:0/3582376900 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc1301649c0 con 0x7fc1300302b0 2024-09-18T07:14:17.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff344001b10 con 0x7ff34c0aff60 2024-09-18T07:14:17.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7fc13759e640 1 -- 172.21.15.70:0/3582376900 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc13002ec10 con 0x7fc1300302b0 2024-09-18T07:14:17.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc128001b00 con 0x7fc1300302b0 2024-09-18T07:14:17.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc128003a40 con 0x7fc1300302b0 2024-09-18T07:14:17.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2054562775 0 0) 0x7ff34c0ab560 con 0x7ff34c074920 2024-09-18T07:14:17.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3102026038 0 0) 0x7f27c0144f50 con 0x7f27c00aadb0 2024-09-18T07:14:17.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff338003520 con 0x7ff34c074920 2024-09-18T07:14:17.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27ac003520 con 0x7f27c00aadb0 2024-09-18T07:14:17.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff340001c00 con 0x7ff34c074920 2024-09-18T07:14:17.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7efeca368640 1 -- 172.21.15.70:0/1188659531 shutdown_connections 2024-09-18T07:14:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7faf08001b20 con 0x7faf10036b90 2024-09-18T07:14:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4b8c001800 con 0x7f4b940740c0 2024-09-18T07:14:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7f4b77fff640 1 -- 172.21.15.70:0/2284101237 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b94072c60 con 0x7f4b940740c0 2024-09-18T07:14:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1393619566 0 0) 0x7f27ac003520 con 0x7f27c00aadb0 2024-09-18T07:14:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f27c0144f50 con 0x7f27c00aadb0 2024-09-18T07:14:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f27b40019c0 con 0x7f27c00aadb0 2024-09-18T07:14:17.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.026+0000 7faf1f3ec640 1 -- 172.21.15.70:0/897662444 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf10038ee0 con 0x7faf10036b90 2024-09-18T07:14:17.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4007035094 0 0) 0x7f27c0144f50 con 0x7f27c00aadb0 2024-09-18T07:14:17.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 >> v1:172.21.15.172:6789/0 conn(0x7f27c00b0e20 legacy=0x7f27c013e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 >> v1:172.21.15.70:6789/0 conn(0x7f27c00ab190 legacy=0x7f27c0138bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c0143b30 con 0x7f27c00aadb0 2024-09-18T07:14:17.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.028+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f4624001240 con 0x7f4634068bf0 2024-09-18T07:14:17.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.028+0000 7f463aa60640 1 -- 172.21.15.70:0/1900369842 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46340b8cb0 con 0x7f4634068bf0 2024-09-18T07:14:17.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.028+0000 7f27c508d640 1 -- 172.21.15.70:0/2809473940 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f27c01392d0 con 0x7f27c00aadb0 2024-09-18T07:14:17.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.028+0000 7f27c508d640 1 -- 172.21.15.70:0/2809473940 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f27c013f110 con 0x7f27c00aadb0 2024-09-18T07:14:17.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.028+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f27b4001b60 con 0x7f27c00aadb0 2024-09-18T07:14:17.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.028+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f27b4003a40 con 0x7f27c00aadb0 2024-09-18T07:14:17.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.028+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f4624095990 con 0x7f4634068bf0 2024-09-18T07:14:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2535771141 0 0) 0x7ff34c137e70 con 0x7ff34c0aff60 2024-09-18T07:14:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 >> v1:172.21.15.70:6790/0 conn(0x7ff34c0714e0 legacy=0x7ff34c1dfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 >> v1:172.21.15.70:6789/0 conn(0x7ff34c074920 legacy=0x7ff34c137050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c143cd0 con 0x7ff34c0aff60 2024-09-18T07:14:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7ff351e17640 1 -- 172.21.15.70:0/720620672 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c1e0460 con 0x7ff34c0aff60 2024-09-18T07:14:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7ff351e17640 1 -- 172.21.15.70:0/720620672 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff34c13baa0 con 0x7ff34c0aff60 2024-09-18T07:14:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff344002040 con 0x7ff34c0aff60 2024-09-18T07:14:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff344003b30 con 0x7ff34c0aff60 2024-09-18T07:14:17.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.028+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f4b8c01c550 con 0x7f4b940740c0 2024-09-18T07:14:17.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.029+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f4b8c094bf0 con 0x7f4b940740c0 2024-09-18T07:14:17.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.029+0000 7efeca368640 1 -- 172.21.15.70:0/1188659531 wait complete. 2024-09-18T07:14:17.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7faf08005010 con 0x7faf10036b90 2024-09-18T07:14:17.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.027+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7faf0801dde0 con 0x7faf10036b90 2024-09-18T07:14:17.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.028+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7faf08096040 con 0x7faf10036b90 2024-09-18T07:14:17.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.030+0000 7fc13759e640 1 -- 172.21.15.70:0/3582376900 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc114005080 con 0x7fc1300302b0 2024-09-18T07:14:17.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.030+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fc128003be0 con 0x7fc1300302b0 2024-09-18T07:14:17.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.028+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7ff240003fe0 con 0x7ff23c0302b0 2024-09-18T07:14:17.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.030+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7ff344003cd0 con 0x7ff34c0aff60 2024-09-18T07:14:17.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.030+0000 7ff351e17640 1 -- 172.21.15.70:0/720620672 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff32c005080 con 0x7ff34c0aff60 2024-09-18T07:14:17.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.030+0000 7f27c508d640 1 -- 172.21.15.70:0/2809473940 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a0005080 con 0x7f27c00aadb0 2024-09-18T07:14:17.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.028+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7ff240095ef0 con 0x7ff23c0302b0 2024-09-18T07:14:17.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.029+0000 7ff24a1f8640 1 -- 172.21.15.70:0/320659012 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff224005080 con 0x7ff23c0302b0 2024-09-18T07:14:17.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.031+0000 7efeca368640 1 Processor -- start 2024-09-18T07:14:17.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.031+0000 7efeca368640 1 -- start start 2024-09-18T07:14:17.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.031+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7fc1280960a0 con 0x7fc1300302b0 2024-09-18T07:14:17.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.031+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7ff344095050 con 0x7ff34c0aff60 2024-09-18T07:14:17.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.031+0000 7efeca368640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec4137e60 con 0x7efec40b15d0 2024-09-18T07:14:17.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.031+0000 7efeca368640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec4135ef0 con 0x7efec4072150 2024-09-18T07:14:17.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.031+0000 7efec9b67640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7efec413a970 0x7efec41e0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37850/0 (socket says 172.21.15.70:37850) 2024-09-18T07:14:17.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.031+0000 7efec9b67640 1 -- 172.21.15.70:0/789887774 learned_addr learned my addr 172.21.15.70:0/789887774 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:17.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.031+0000 7efeca368640 1 -- 172.21.15.70:0/789887774 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec4136600 con 0x7efec413a970 2024-09-18T07:14:17.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.033+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f4b941402b0 con 0x7f4b940740c0 2024-09-18T07:14:17.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.035+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f462405e300 con 0x7f4634068bf0 2024-09-18T07:14:17.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.035+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2509089016 0 0) 0x7efec4137e60 con 0x7efec40b15d0 2024-09-18T07:14:17.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.035+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efea8003520 con 0x7efec40b15d0 2024-09-18T07:14:17.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.035+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7faf0805eaf0 con 0x7faf10036b90 2024-09-18T07:14:17.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.035+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2681994633 0 0) 0x7efea8003520 con 0x7efec40b15d0 2024-09-18T07:14:17.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.035+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efec4137e60 con 0x7efec40b15d0 2024-09-18T07:14:17.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.035+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7efeb4001870 con 0x7efec40b15d0 2024-09-18T07:14:17.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.035+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 38032957 0 0) 0x7efec4136600 con 0x7efec413a970 2024-09-18T07:14:17.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.035+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efea8003520 con 0x7efec413a970 2024-09-18T07:14:17.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.037+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f27b4003be0 con 0x7f27c00aadb0 2024-09-18T07:14:17.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.037+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 347603603 0 0) 0x7efec4137e60 con 0x7efec40b15d0 2024-09-18T07:14:17.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.037+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 >> v1:172.21.15.70:6790/0 conn(0x7efec413a970 legacy=0x7efec41e0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.037+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 >> v1:172.21.15.172:6789/0 conn(0x7efec4072150 legacy=0x7efec4143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.037+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec41367d0 con 0x7efec40b15d0 2024-09-18T07:14:17.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.037+0000 7efeca368640 1 -- 172.21.15.70:0/789887774 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7efec4135830 con 0x7efec40b15d0 2024-09-18T07:14:17.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.037+0000 7efeca368640 1 -- 172.21.15.70:0/789887774 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7efec406cb50 con 0x7efec40b15d0 2024-09-18T07:14:17.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.037+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7ff24005e860 con 0x7ff23c0302b0 2024-09-18T07:14:17.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.038+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efeb4003690 con 0x7efec40b15d0 2024-09-18T07:14:17.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.038+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7efeb40039e0 con 0x7efec40b15d0 2024-09-18T07:14:17.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.039+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f27b4096060 con 0x7f27c00aadb0 2024-09-18T07:14:17.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.040+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7efeb401c7b0 con 0x7efec40b15d0 2024-09-18T07:14:17.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.040+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f27b405e9d0 con 0x7f27c00aadb0 2024-09-18T07:14:17.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.041+0000 7efeaf7fe640 1 -- 172.21.15.70:0/789887774 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec41439c0 con 0x7efec40b15d0 2024-09-18T07:14:17.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.041+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7ff34405d9c0 con 0x7ff34c0aff60 2024-09-18T07:14:17.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.041+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7efeb4094da0 con 0x7efec40b15d0 2024-09-18T07:14:17.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.046+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7fc12805ea10 con 0x7fc1300302b0 2024-09-18T07:14:17.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.050+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7efeb405d710 con 0x7efec40b15d0 2024-09-18T07:14:17.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.450+0000 7f463aa60640 1 -- 172.21.15.70:0/1900369842 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f463406cb50 con 0x7f4634068bf0 2024-09-18T07:14:17.455 INFO:teuthology.orchestra.run.smithi070.stdout:60129542147 2024-09-18T07:14:17.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.451+0000 7f4630ff9640 1 -- 172.21.15.70:0/1900369842 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (unknown 92182286 0 13373408) 0x7f462405e300 con 0x7f4634068bf0 2024-09-18T07:14:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.452+0000 7f4b77fff640 1 -- 172.21.15.70:0/2284101237 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4b940698c0 con 0x7f4b940740c0 2024-09-18T07:14:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.453+0000 7f4b8a7fc640 1 -- 172.21.15.70:0/2284101237 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (unknown 1482059429 0 2194451396) 0x7f4b8c05d560 con 0x7f4b940740c0 2024-09-18T07:14:17.456 INFO:teuthology.orchestra.run.smithi070.stdout:55834574851 2024-09-18T07:14:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.455+0000 7f463aa60640 1 -- 172.21.15.70:0/1900369842 >> v1:172.21.15.70:6816/1657990791 conn(0x7f461c07a460 legacy=0x7f461c07c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.455+0000 7f463aa60640 1 -- 172.21.15.70:0/1900369842 >> v1:172.21.15.70:6790/0 conn(0x7f4634068bf0 legacy=0x7f4634134220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.456+0000 7f463aa60640 1 -- 172.21.15.70:0/1900369842 shutdown_connections 2024-09-18T07:14:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.456+0000 7f463aa60640 1 -- 172.21.15.70:0/1900369842 >> 172.21.15.70:0/1900369842 conn(0x7f46340b9440 msgr2=0x7f46340b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.456+0000 7f463aa60640 1 -- 172.21.15.70:0/1900369842 shutdown_connections 2024-09-18T07:14:17.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.456+0000 7f463aa60640 1 -- 172.21.15.70:0/1900369842 wait complete. 2024-09-18T07:14:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.457+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2284101237 >> v1:172.21.15.70:6816/1657990791 conn(0x7f4b78079dd0 legacy=0x7f4b7807c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.457+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2284101237 >> v1:172.21.15.172:6789/0 conn(0x7f4b940740c0 legacy=0x7f4b9413e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.457+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2284101237 shutdown_connections 2024-09-18T07:14:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.457+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2284101237 >> 172.21.15.70:0/2284101237 conn(0x7f4b940b4a40 msgr2=0x7f4b940b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.457+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2284101237 shutdown_connections 2024-09-18T07:14:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.457+0000 7f4b9aa00640 1 -- 172.21.15.70:0/2284101237 wait complete. 2024-09-18T07:14:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.458+0000 7faf1f3ec640 1 -- 172.21.15.70:0/897662444 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7faf100090f0 con 0x7faf10036b90 2024-09-18T07:14:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.459+0000 7faf0dffb640 1 -- 172.21.15.70:0/897662444 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (unknown 574334944 0 13373408) 0x7faf080018c0 con 0x7faf10036b90 2024-09-18T07:14:17.461 INFO:teuthology.orchestra.run.smithi070.stdout:60129542147 2024-09-18T07:14:17.465 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.462+0000 7faf037fe640 1 -- 172.21.15.70:0/897662444 >> v1:172.21.15.70:6816/1657990791 conn(0x7faefc079820 legacy=0x7faefc07bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.465 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.463+0000 7faf037fe640 1 -- 172.21.15.70:0/897662444 >> v1:172.21.15.70:6789/0 conn(0x7faf10036b90 legacy=0x7faf100a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.463+0000 7ff24a1f8640 1 -- 172.21.15.70:0/320659012 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff224005370 con 0x7ff23c0302b0 2024-09-18T07:14:17.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.463+0000 7faf037fe640 1 -- 172.21.15.70:0/897662444 shutdown_connections 2024-09-18T07:14:17.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.463+0000 7faf037fe640 1 -- 172.21.15.70:0/897662444 >> 172.21.15.70:0/897662444 conn(0x7faf10039d60 msgr2=0x7faf10037b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.463+0000 7faf037fe640 1 -- 172.21.15.70:0/897662444 shutdown_connections 2024-09-18T07:14:17.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.464+0000 7faf037fe640 1 -- 172.21.15.70:0/897662444 wait complete. 2024-09-18T07:14:17.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.464+0000 7ff239ffb640 1 -- 172.21.15.70:0/320659012 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (unknown 1823589948 0 13373408) 0x7ff24005e160 con 0x7ff23c0302b0 2024-09-18T07:14:17.467 INFO:teuthology.orchestra.run.smithi070.stdout:60129542147 2024-09-18T07:14:17.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.465+0000 7f27c508d640 1 -- 172.21.15.70:0/2809473940 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f27a0005370 con 0x7f27c00aadb0 2024-09-18T07:14:17.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.465+0000 7ff351e17640 1 -- 172.21.15.70:0/720620672 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff32c005370 con 0x7ff34c0aff60 2024-09-18T07:14:17.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.466+0000 7f27bcff9640 1 -- 172.21.15.70:0/2809473940 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (unknown 383520633 0 13373408) 0x7f27b405e2d0 con 0x7f27c00aadb0 2024-09-18T07:14:17.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.466+0000 7ff3497fa640 1 -- 172.21.15.70:0/720620672 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (unknown 2131975755 0 2194451396) 0x7ff34405d2c0 con 0x7ff34c0aff60 2024-09-18T07:14:17.469 INFO:teuthology.orchestra.run.smithi070.stdout:55834574851 2024-09-18T07:14:17.470 INFO:teuthology.orchestra.run.smithi070.stdout:60129542147 2024-09-18T07:14:17.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.468+0000 7ff24a1f8640 1 -- 172.21.15.70:0/320659012 >> v1:172.21.15.70:6816/1657990791 conn(0x7ff228079a00 legacy=0x7ff22807bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.468+0000 7ff24a1f8640 1 -- 172.21.15.70:0/320659012 >> v1:172.21.15.172:6789/0 conn(0x7ff23c0302b0 legacy=0x7ff23c160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.469+0000 7ff24a1f8640 1 -- 172.21.15.70:0/320659012 shutdown_connections 2024-09-18T07:14:17.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.469+0000 7ff24a1f8640 1 -- 172.21.15.70:0/320659012 >> 172.21.15.70:0/320659012 conn(0x7ff23c005460 msgr2=0x7ff23c0d76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.469+0000 7ff24a1f8640 1 -- 172.21.15.70:0/320659012 shutdown_connections 2024-09-18T07:14:17.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.469+0000 7ff24a1f8640 1 -- 172.21.15.70:0/320659012 wait complete. 2024-09-18T07:14:17.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.470+0000 7f27c508d640 1 -- 172.21.15.70:0/2809473940 >> v1:172.21.15.70:6816/1657990791 conn(0x7f27ac0797f0 legacy=0x7f27ac07bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.470+0000 7ff351e17640 1 -- 172.21.15.70:0/720620672 >> v1:172.21.15.70:6816/1657990791 conn(0x7ff338079950 legacy=0x7ff33807be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.470+0000 7ff351e17640 1 -- 172.21.15.70:0/720620672 >> v1:172.21.15.172:6789/0 conn(0x7ff34c0aff60 legacy=0x7ff34c137760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.470+0000 7fc13759e640 1 -- 172.21.15.70:0/3582376900 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc114005370 con 0x7fc1300302b0 2024-09-18T07:14:17.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.470+0000 7f27c508d640 1 -- 172.21.15.70:0/2809473940 >> v1:172.21.15.70:6790/0 conn(0x7f27c00aadb0 legacy=0x7f27c0144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.470+0000 7ff351e17640 1 -- 172.21.15.70:0/720620672 shutdown_connections 2024-09-18T07:14:17.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.470+0000 7ff351e17640 1 -- 172.21.15.70:0/720620672 >> 172.21.15.70:0/720620672 conn(0x7ff34c0b5b50 msgr2=0x7ff34c0b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.471+0000 7ff351e17640 1 -- 172.21.15.70:0/720620672 shutdown_connections 2024-09-18T07:14:17.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.471+0000 7ff351e17640 1 -- 172.21.15.70:0/720620672 wait complete. 2024-09-18T07:14:17.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.470+0000 7fc12effd640 1 -- 172.21.15.70:0/3582376900 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (unknown 832126871 0 13373408) 0x7fc12805e310 con 0x7fc1300302b0 2024-09-18T07:14:17.475 INFO:teuthology.orchestra.run.smithi070.stdout:60129542147 2024-09-18T07:14:17.475 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.471+0000 7f27c508d640 1 -- 172.21.15.70:0/2809473940 shutdown_connections 2024-09-18T07:14:17.475 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.471+0000 7f27c508d640 1 -- 172.21.15.70:0/2809473940 >> 172.21.15.70:0/2809473940 conn(0x7f27c00b4a40 msgr2=0x7f27c013cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.476 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.471+0000 7f27c508d640 1 -- 172.21.15.70:0/2809473940 shutdown_connections 2024-09-18T07:14:17.476 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.471+0000 7f27c508d640 1 -- 172.21.15.70:0/2809473940 wait complete. 2024-09-18T07:14:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.475+0000 7fc13759e640 1 -- 172.21.15.70:0/3582376900 >> v1:172.21.15.70:6816/1657990791 conn(0x7fc1200796f0 legacy=0x7fc12007bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.475+0000 7fc13759e640 1 -- 172.21.15.70:0/3582376900 >> v1:172.21.15.172:6789/0 conn(0x7fc1300302b0 legacy=0x7fc13002d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.475+0000 7fc13759e640 1 -- 172.21.15.70:0/3582376900 shutdown_connections 2024-09-18T07:14:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.475+0000 7fc13759e640 1 -- 172.21.15.70:0/3582376900 >> 172.21.15.70:0/3582376900 conn(0x7fc130005460 msgr2=0x7fc1300367e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.475+0000 7efeaf7fe640 1 -- 172.21.15.70:0/789887774 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7efec413ae00 con 0x7efec40b15d0 2024-09-18T07:14:17.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.475+0000 7fc13759e640 1 -- 172.21.15.70:0/3582376900 shutdown_connections 2024-09-18T07:14:17.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.475+0000 7fc13759e640 1 -- 172.21.15.70:0/3582376900 wait complete. 2024-09-18T07:14:17.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.476+0000 7efeb9ffb640 1 -- 172.21.15.70:0/789887774 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (unknown 1274345170 0 2194451396) 0x7efeb405d010 con 0x7efec40b15d0 2024-09-18T07:14:17.480 INFO:teuthology.orchestra.run.smithi070.stdout:55834574851 2024-09-18T07:14:17.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.2 2024-09-18T07:14:17.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.7 2024-09-18T07:14:17.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.479+0000 7efeaf7fe640 1 -- 172.21.15.70:0/789887774 >> v1:172.21.15.70:6816/1657990791 conn(0x7efea807dcf0 legacy=0x7efea80801b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.479+0000 7efeaf7fe640 1 -- 172.21.15.70:0/789887774 >> v1:172.21.15.70:6789/0 conn(0x7efec40b15d0 legacy=0x7efec41444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:17.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.479+0000 7efeaf7fe640 1 -- 172.21.15.70:0/789887774 shutdown_connections 2024-09-18T07:14:17.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.479+0000 7efeaf7fe640 1 -- 172.21.15.70:0/789887774 >> 172.21.15.70:0/789887774 conn(0x7efec40b6bd0 msgr2=0x7efec413ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:17.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.479+0000 7efeaf7fe640 1 -- 172.21.15.70:0/789887774 shutdown_connections 2024-09-18T07:14:17.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:17.479+0000 7efeaf7fe640 1 -- 172.21.15.70:0/789887774 wait complete. 2024-09-18T07:14:17.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.0 2024-09-18T07:14:17.491 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.4 2024-09-18T07:14:17.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.5 2024-09-18T07:14:17.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.3 2024-09-18T07:14:17.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.1 2024-09-18T07:14:17.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.6 2024-09-18T07:14:18.481 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T07:14:18.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T07:14:18.489 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T07:14:18.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T07:14:18.496 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T07:14:18.498 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T07:14:18.498 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T07:14:18.498 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T07:14:18.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.627+0000 7f766a0ca640 1 Processor -- start 2024-09-18T07:14:18.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.627+0000 7f766a0ca640 1 -- start start 2024-09-18T07:14:18.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.628+0000 7f766a0ca640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765c02f220 con 0x7f765c0306d0 2024-09-18T07:14:18.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.628+0000 7f766a0ca640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765c030e60 con 0x7f765c0c65c0 2024-09-18T07:14:18.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.628+0000 7f766a0ca640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765c032180 con 0x7f765c0064d0 2024-09-18T07:14:18.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.629+0000 7f76690c8640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f765c0064d0 0x7f765c031340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37866/0 (socket says 172.21.15.70:37866) 2024-09-18T07:14:18.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.629+0000 7f76690c8640 1 -- 172.21.15.70:0/3354248592 learned_addr learned my addr 172.21.15.70:0/3354248592 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.629+0000 7f765bfff640 1 -- 172.21.15.70:0/3354248592 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1590201159 0 0) 0x7f765c032180 con 0x7f765c0064d0 2024-09-18T07:14:18.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.629+0000 7f765bfff640 1 -- 172.21.15.70:0/3354248592 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7650003520 con 0x7f765c0064d0 2024-09-18T07:14:18.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.630+0000 7f765bfff640 1 -- 172.21.15.70:0/3354248592 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4170266149 0 0) 0x7f7660001040 con 0x7f765c0306d0 2024-09-18T07:14:18.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.630+0000 7f765bfff640 1 -- 172.21.15.70:0/3354248592 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f765c032180 con 0x7f765c0306d0 2024-09-18T07:14:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.630+0000 7f765bfff640 1 -- 172.21.15.70:0/3354248592 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2188605644 0 0) 0x7f765c030e60 con 0x7f765c0c65c0 2024-09-18T07:14:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.630+0000 7f765bfff640 1 -- 172.21.15.70:0/3354248592 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7660001040 con 0x7f765c0c65c0 2024-09-18T07:14:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.630+0000 7f765bfff640 1 -- 172.21.15.70:0/3354248592 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3296305161 0 0) 0x7f765c032180 con 0x7f765c0306d0 2024-09-18T07:14:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.630+0000 7f765bfff640 1 -- 172.21.15.70:0/3354248592 >> v1:172.21.15.70:6790/0 conn(0x7f765c0064d0 legacy=0x7f765c031340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.630+0000 7f765bfff640 1 -- 172.21.15.70:0/3354248592 >> v1:172.21.15.172:6789/0 conn(0x7f765c0c65c0 legacy=0x7f765c0ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.631+0000 7f765bfff640 1 -- 172.21.15.70:0/3354248592 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765c032460 con 0x7f765c0306d0 2024-09-18T07:14:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.631+0000 7f765bfff640 1 -- 172.21.15.70:0/3354248592 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7660001710 con 0x7f765c0306d0 2024-09-18T07:14:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.631+0000 7f765bfff640 1 -- 172.21.15.70:0/3354248592 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7660001970 con 0x7f765c0306d0 2024-09-18T07:14:18.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.631+0000 7f765bfff640 1 -- 172.21.15.70:0/3354248592 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7660001cc0 con 0x7f765c0306d0 2024-09-18T07:14:18.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.632+0000 7f766a0ca640 1 -- 172.21.15.70:0/3354248592 >> v1:172.21.15.70:6789/0 conn(0x7f765c0306d0 legacy=0x7f765c0c0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.633+0000 7fc66fc2a640 1 Processor -- start 2024-09-18T07:14:18.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.632+0000 7f766a0ca640 1 -- 172.21.15.70:0/3354248592 shutdown_connections 2024-09-18T07:14:18.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.632+0000 7f766a0ca640 1 -- 172.21.15.70:0/3354248592 >> 172.21.15.70:0/3354248592 conn(0x7f765c005460 msgr2=0x7f765c005870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:18.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.633+0000 7fc66fc2a640 1 -- start start 2024-09-18T07:14:18.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.634+0000 7fc66fc2a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6600327b0 con 0x7fc6600081f0 2024-09-18T07:14:18.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.634+0000 7fc66fc2a640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6600318a0 con 0x7fc6600067e0 2024-09-18T07:14:18.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.634+0000 7fc66fc2a640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc660032a20 con 0x7fc6600cbc10 2024-09-18T07:14:18.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.634+0000 7fc66d9c6640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc6600067e0 0x7fc6600343b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49832/0 (socket says 172.21.15.70:49832) 2024-09-18T07:14:18.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.633+0000 7f766a0ca640 1 -- 172.21.15.70:0/3354248592 shutdown_connections 2024-09-18T07:14:18.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.634+0000 7fc66d9c6640 1 -- 172.21.15.70:0/2984524945 learned_addr learned my addr 172.21.15.70:0/2984524945 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.635+0000 7fc66c9c4640 1 -- 172.21.15.70:0/2984524945 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3007416966 0 0) 0x7fc6600318a0 con 0x7fc6600067e0 2024-09-18T07:14:18.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.635+0000 7fc66c9c4640 1 -- 172.21.15.70:0/2984524945 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc654003520 con 0x7fc6600067e0 2024-09-18T07:14:18.640 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.633+0000 7f766a0ca640 1 -- 172.21.15.70:0/3354248592 wait complete. 2024-09-18T07:14:18.640 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.634+0000 7f766a0ca640 1 Processor -- start 2024-09-18T07:14:18.640 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.634+0000 7f766a0ca640 1 -- start start 2024-09-18T07:14:18.640 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.636+0000 7fc66c9c4640 1 -- 172.21.15.70:0/2984524945 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2682383276 0 0) 0x7fc654003520 con 0x7fc6600067e0 2024-09-18T07:14:18.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.636+0000 7f76690c8640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f765c0064d0 0x7f765c02b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37888/0 (socket says 172.21.15.70:37888) 2024-09-18T07:14:18.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.636+0000 7fc66c9c4640 1 -- 172.21.15.70:0/2984524945 >> v1:172.21.15.70:6790/0 conn(0x7fc6600cbc10 legacy=0x7fc6600d6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.636+0000 7f76690c8640 1 -- 172.21.15.70:0/3894634665 learned_addr learned my addr 172.21.15.70:0/3894634665 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.636+0000 7fc66c9c4640 1 -- 172.21.15.70:0/2984524945 >> v1:172.21.15.70:6789/0 conn(0x7fc6600081f0 legacy=0x7fc6600cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.637+0000 7fc66c9c4640 1 -- 172.21.15.70:0/2984524945 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc660008ec0 con 0x7fc6600067e0 2024-09-18T07:14:18.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.637+0000 7fc66c9c4640 1 -- 172.21.15.70:0/2984524945 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc664001690 con 0x7fc6600067e0 2024-09-18T07:14:18.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.637+0000 7f766a0ca640 1 -- 172.21.15.70:0/3894634665 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765c02a960 con 0x7f765c0306d0 2024-09-18T07:14:18.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.637+0000 7fc66c9c4640 1 -- 172.21.15.70:0/2984524945 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc6640018f0 con 0x7fc6600067e0 2024-09-18T07:14:18.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.637+0000 7f766a0ca640 1 -- 172.21.15.70:0/3894634665 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765c0b71e0 con 0x7f765c0c65c0 2024-09-18T07:14:18.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.637+0000 7f766a0ca640 1 -- 172.21.15.70:0/3894634665 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765c02abd0 con 0x7f765c0064d0 2024-09-18T07:14:18.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.638+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1461430926 0 0) 0x7f765c02a960 con 0x7f765c0306d0 2024-09-18T07:14:18.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.638+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7648003520 con 0x7f765c0306d0 2024-09-18T07:14:18.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.638+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2984524945 >> v1:172.21.15.172:6789/0 conn(0x7fc6600067e0 legacy=0x7fc6600343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.638+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2984524945 shutdown_connections 2024-09-18T07:14:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.638+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2984524945 >> 172.21.15.70:0/2984524945 conn(0x7fc660005840 msgr2=0x7fc660005c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.639+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2863108780 0 0) 0x7f765c02abd0 con 0x7f765c0064d0 2024-09-18T07:14:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.639+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f765c02a960 con 0x7f765c0064d0 2024-09-18T07:14:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.639+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3239866986 0 0) 0x7f7648003520 con 0x7f765c0306d0 2024-09-18T07:14:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.639+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f765c02abd0 con 0x7f765c0306d0 2024-09-18T07:14:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.640+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4088661957 0 0) 0x7f765c02a960 con 0x7f765c0064d0 2024-09-18T07:14:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.640+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7648003520 con 0x7f765c0064d0 2024-09-18T07:14:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.640+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f76600015d0 con 0x7f765c0306d0 2024-09-18T07:14:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.640+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7664051dd0 con 0x7f765c0064d0 2024-09-18T07:14:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.641+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 626061030 0 0) 0x7f765c02abd0 con 0x7f765c0306d0 2024-09-18T07:14:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.641+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 >> v1:172.21.15.70:6790/0 conn(0x7f765c0064d0 legacy=0x7f765c02b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.641+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 >> v1:172.21.15.172:6789/0 conn(0x7f765c0c65c0 legacy=0x7f765c0b6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.641+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765c0c0290 con 0x7f765c0306d0 2024-09-18T07:14:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.641+0000 7f766a0ca640 1 -- 172.21.15.70:0/3894634665 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f765c02b770 con 0x7f765c0306d0 2024-09-18T07:14:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.639+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2984524945 shutdown_connections 2024-09-18T07:14:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.639+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2984524945 wait complete. 2024-09-18T07:14:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.640+0000 7fc66fc2a640 1 Processor -- start 2024-09-18T07:14:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.640+0000 7fc66fc2a640 1 -- start start 2024-09-18T07:14:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.641+0000 7fc66fc2a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6600a44b0 con 0x7fc6600081f0 2024-09-18T07:14:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.641+0000 7fc66fc2a640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6600a2bb0 con 0x7fc6600067e0 2024-09-18T07:14:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.641+0000 7fc66fc2a640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6600cac30 con 0x7fc660030c00 2024-09-18T07:14:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.642+0000 7fc66d9c6640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc6600067e0 0x7fc66002ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49848/0 (socket says 172.21.15.70:49848) 2024-09-18T07:14:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.642+0000 7fc66d9c6640 1 -- 172.21.15.70:0/2308365993 learned_addr learned my addr 172.21.15.70:0/2308365993 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.644+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1328594872 0 0) 0x7fc6600a2bb0 con 0x7fc6600067e0 2024-09-18T07:14:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.644+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc64c003520 con 0x7fc6600067e0 2024-09-18T07:14:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.645+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3564924330 0 0) 0x7fc6600a44b0 con 0x7fc6600081f0 2024-09-18T07:14:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.645+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6600a2bb0 con 0x7fc6600081f0 2024-09-18T07:14:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.645+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1503084795 0 0) 0x7fc6600cac30 con 0x7fc660030c00 2024-09-18T07:14:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.645+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6600a44b0 con 0x7fc660030c00 2024-09-18T07:14:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.645+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2515112556 0 0) 0x7fc64c003520 con 0x7fc6600067e0 2024-09-18T07:14:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.645+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc6600cac30 con 0x7fc6600067e0 2024-09-18T07:14:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.645+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc664004a80 con 0x7fc6600067e0 2024-09-18T07:14:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.645+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1185725718 0 0) 0x7fc6600a2bb0 con 0x7fc6600081f0 2024-09-18T07:14:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.645+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc64c003520 con 0x7fc6600081f0 2024-09-18T07:14:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.646+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc658001790 con 0x7fc6600081f0 2024-09-18T07:14:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.646+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3349159499 0 0) 0x7fc6600cac30 con 0x7fc6600067e0 2024-09-18T07:14:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.646+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 >> v1:172.21.15.70:6790/0 conn(0x7fc660030c00 legacy=0x7fc6600a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.646+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 >> v1:172.21.15.70:6789/0 conn(0x7fc6600081f0 legacy=0x7fc6600a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.646+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6600cae00 con 0x7fc6600067e0 2024-09-18T07:14:18.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.643+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f76600038a0 con 0x7f765c0306d0 2024-09-18T07:14:18.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.646+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2308365993 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc660186560 con 0x7fc6600067e0 2024-09-18T07:14:18.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.646+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2308365993 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc660186910 con 0x7fc6600067e0 2024-09-18T07:14:18.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.643+0000 7f766a0ca640 1 -- 172.21.15.70:0/3894634665 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f765c0ba440 con 0x7f765c0306d0 2024-09-18T07:14:18.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.646+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc664001870 con 0x7fc6600067e0 2024-09-18T07:14:18.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.646+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc664001cc0 con 0x7fc6600067e0 2024-09-18T07:14:18.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.643+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7660003bf0 con 0x7f765c0306d0 2024-09-18T07:14:18.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.648+0000 7f766a0ca640 1 -- 172.21.15.70:0/3894634665 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7644005080 con 0x7f765c0306d0 2024-09-18T07:14:18.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.649+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f766001c9c0 con 0x7f765c0306d0 2024-09-18T07:14:18.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.650+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f7660094c80 con 0x7f765c0306d0 2024-09-18T07:14:18.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.651+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fc66401db00 con 0x7fc6600067e0 2024-09-18T07:14:18.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.652+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7fc664095dd0 con 0x7fc6600067e0 2024-09-18T07:14:18.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.653+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2308365993 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc650005080 con 0x7fc6600067e0 2024-09-18T07:14:18.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.658+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7fc66405e740 con 0x7fc6600067e0 2024-09-18T07:14:18.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.658+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f765c02f220 con 0x7f765c0306d0 2024-09-18T07:14:18.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.687+0000 7f82506c6640 1 Processor -- start 2024-09-18T07:14:18.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.687+0000 7f82506c6640 1 -- start start 2024-09-18T07:14:18.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.688+0000 7f82506c6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8248144ab0 con 0x7f82481452e0 2024-09-18T07:14:18.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.688+0000 7f82506c6640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82480b2390 con 0x7f82480729a0 2024-09-18T07:14:18.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.688+0000 7f82506c6640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f824813e960 con 0x7f8248073b60 2024-09-18T07:14:18.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.688+0000 7f824dc61640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f82481452e0 0x7f824813e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52060/0 (socket says 172.21.15.70:52060) 2024-09-18T07:14:18.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.688+0000 7f824dc61640 1 -- 172.21.15.70:0/1879230303 learned_addr learned my addr 172.21.15.70:0/1879230303 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.688+0000 7f824d460640 1 -- 172.21.15.70:0/1879230303 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 592836428 0 0) 0x7f824813e960 con 0x7f8248073b60 2024-09-18T07:14:18.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.689+0000 7f824d460640 1 -- 172.21.15.70:0/1879230303 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8234003520 con 0x7f8248073b60 2024-09-18T07:14:18.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.689+0000 7f824d460640 1 -- 172.21.15.70:0/1879230303 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342683743 0 0) 0x7f82480b2390 con 0x7f82480729a0 2024-09-18T07:14:18.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.689+0000 7f824d460640 1 -- 172.21.15.70:0/1879230303 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f824813e960 con 0x7f82480729a0 2024-09-18T07:14:18.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.689+0000 7f824d460640 1 -- 172.21.15.70:0/1879230303 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1503208606 0 0) 0x7f8248144ab0 con 0x7f82481452e0 2024-09-18T07:14:18.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.689+0000 7f824d460640 1 -- 172.21.15.70:0/1879230303 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82480b2390 con 0x7f82481452e0 2024-09-18T07:14:18.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.689+0000 7f824d460640 1 -- 172.21.15.70:0/1879230303 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3904664044 0 0) 0x7f824813e960 con 0x7f82480729a0 2024-09-18T07:14:18.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.689+0000 7f824d460640 1 -- 172.21.15.70:0/1879230303 >> v1:172.21.15.70:6790/0 conn(0x7f8248073b60 legacy=0x7f824814c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.689+0000 7f824d460640 1 -- 172.21.15.70:0/1879230303 >> v1:172.21.15.70:6789/0 conn(0x7f82481452e0 legacy=0x7f824813e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.689+0000 7f824d460640 1 -- 172.21.15.70:0/1879230303 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824813eb30 con 0x7f82480729a0 2024-09-18T07:14:18.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.689+0000 7f824d460640 1 -- 172.21.15.70:0/1879230303 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8240001920 con 0x7f82480729a0 2024-09-18T07:14:18.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.689+0000 7f824d460640 1 -- 172.21.15.70:0/1879230303 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8240001b80 con 0x7f82480729a0 2024-09-18T07:14:18.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.689+0000 7f824d460640 1 -- 172.21.15.70:0/1879230303 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8240001f50 con 0x7f82480729a0 2024-09-18T07:14:18.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.690+0000 7f82506c6640 1 -- 172.21.15.70:0/1879230303 >> v1:172.21.15.172:6789/0 conn(0x7f82480729a0 legacy=0x7f82480b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.690+0000 7fb756d2c640 1 Processor -- start 2024-09-18T07:14:18.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.690+0000 7f82506c6640 1 -- 172.21.15.70:0/1879230303 shutdown_connections 2024-09-18T07:14:18.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.690+0000 7f82506c6640 1 -- 172.21.15.70:0/1879230303 >> 172.21.15.70:0/1879230303 conn(0x7f82480b3c90 msgr2=0x7f82480b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:18.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.690+0000 7fb756d2c640 1 -- start start 2024-09-18T07:14:18.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.691+0000 7fb754ac8640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb750071bb0 0x7fb75013c4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52070/0 (socket says 172.21.15.70:52070) 2024-09-18T07:14:18.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.692+0000 7fb754ac8640 1 -- 172.21.15.70:0/19381387 learned_addr learned my addr 172.21.15.70:0/19381387 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.692+0000 7fb756d2c640 1 -- 172.21.15.70:0/19381387 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb750072c20 con 0x7fb750071bb0 2024-09-18T07:14:18.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.692+0000 7fb756d2c640 1 -- 172.21.15.70:0/19381387 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb750072e90 con 0x7fb7500710d0 2024-09-18T07:14:18.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.692+0000 7fb756d2c640 1 -- 172.21.15.70:0/19381387 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb750073590 con 0x7fb75013cbe0 2024-09-18T07:14:18.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.691+0000 7f94c6189640 1 Processor -- start 2024-09-18T07:14:18.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.691+0000 7f94c6189640 1 -- start start 2024-09-18T07:14:18.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.692+0000 7f94bf7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f94c00b2a00 0x7f94c013ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49860/0 (socket says 172.21.15.70:49860) 2024-09-18T07:14:18.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.692+0000 7f94bf7fe640 1 -- 172.21.15.70:0/3696179262 learned_addr learned my addr 172.21.15.70:0/3696179262 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.690+0000 7f82506c6640 1 -- 172.21.15.70:0/1879230303 shutdown_connections 2024-09-18T07:14:18.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.691+0000 7f82506c6640 1 -- 172.21.15.70:0/1879230303 wait complete. 2024-09-18T07:14:18.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.692+0000 7f94c6189640 1 -- 172.21.15.70:0/3696179262 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c0073d60 con 0x7f94c0074fb0 2024-09-18T07:14:18.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.692+0000 7f82506c6640 1 Processor -- start 2024-09-18T07:14:18.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.692+0000 7f82506c6640 1 -- start start 2024-09-18T07:14:18.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.692+0000 7f94c6189640 1 -- 172.21.15.70:0/3696179262 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c0071150 con 0x7f94c00b2a00 2024-09-18T07:14:18.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.693+0000 7f94c6189640 1 -- 172.21.15.70:0/3696179262 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c0071320 con 0x7f94c013d150 2024-09-18T07:14:18.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.693+0000 7f82506c6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82480ad790 con 0x7f82481452e0 2024-09-18T07:14:18.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.693+0000 7f82506c6640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82480ada70 con 0x7f8248073b60 2024-09-18T07:14:18.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.693+0000 7f82506c6640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82480ac750 con 0x7f82480729a0 2024-09-18T07:14:18.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.693+0000 7f94be7fc640 1 -- 172.21.15.70:0/3696179262 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3177383347 0 0) 0x7f94c0071150 con 0x7f94c00b2a00 2024-09-18T07:14:18.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.693+0000 7f94be7fc640 1 -- 172.21.15.70:0/3696179262 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94a8003440 con 0x7f94c00b2a00 2024-09-18T07:14:18.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.693+0000 7f94be7fc640 1 -- 172.21.15.70:0/3696179262 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1756781816 0 0) 0x7f94c0073d60 con 0x7f94c0074fb0 2024-09-18T07:14:18.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.693+0000 7f94be7fc640 1 -- 172.21.15.70:0/3696179262 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94c0071150 con 0x7f94c0074fb0 2024-09-18T07:14:18.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.693+0000 7f824ec63640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f82481452e0 0x7f8248201210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52088/0 (socket says 172.21.15.70:52088) 2024-09-18T07:14:18.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.693+0000 7f824ec63640 1 -- 172.21.15.70:0/2240583486 learned_addr learned my addr 172.21.15.70:0/2240583486 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7fb74f7fe640 1 -- 172.21.15.70:0/19381387 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 247080506 0 0) 0x7fb750072e90 con 0x7fb7500710d0 2024-09-18T07:14:18.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7fb74f7fe640 1 -- 172.21.15.70:0/19381387 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb738003520 con 0x7fb7500710d0 2024-09-18T07:14:18.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3211233694 0 0) 0x7f82480ad790 con 0x7f82481452e0 2024-09-18T07:14:18.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f822c003520 con 0x7f82481452e0 2024-09-18T07:14:18.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7fb74f7fe640 1 -- 172.21.15.70:0/19381387 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3547048162 0 0) 0x7fb750073590 con 0x7fb75013cbe0 2024-09-18T07:14:18.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7fb74f7fe640 1 -- 172.21.15.70:0/19381387 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb750072e90 con 0x7fb75013cbe0 2024-09-18T07:14:18.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7fb74f7fe640 1 -- 172.21.15.70:0/19381387 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 479094269 0 0) 0x7fb750072c20 con 0x7fb750071bb0 2024-09-18T07:14:18.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7fb74f7fe640 1 -- 172.21.15.70:0/19381387 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb750073590 con 0x7fb750071bb0 2024-09-18T07:14:18.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f8d735b4640 1 Processor -- start 2024-09-18T07:14:18.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f94be7fc640 1 -- 172.21.15.70:0/3696179262 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1259993066 0 0) 0x7f94c0071320 con 0x7f94c013d150 2024-09-18T07:14:18.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f94be7fc640 1 -- 172.21.15.70:0/3696179262 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94c0073d60 con 0x7f94c013d150 2024-09-18T07:14:18.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f94be7fc640 1 -- 172.21.15.70:0/3696179262 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 269068229 0 0) 0x7f94c0071150 con 0x7f94c0074fb0 2024-09-18T07:14:18.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f94be7fc640 1 -- 172.21.15.70:0/3696179262 >> v1:172.21.15.70:6790/0 conn(0x7f94c013d150 legacy=0x7f94c00723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f94be7fc640 1 -- 172.21.15.70:0/3696179262 >> v1:172.21.15.172:6789/0 conn(0x7f94c00b2a00 legacy=0x7f94c013ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2732723959 0 0) 0x7f82480ada70 con 0x7f8248073b60 2024-09-18T07:14:18.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82480ad790 con 0x7f8248073b60 2024-09-18T07:14:18.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7fb74f7fe640 1 -- 172.21.15.70:0/19381387 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1821828451 0 0) 0x7fb738003520 con 0x7fb7500710d0 2024-09-18T07:14:18.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7fb74f7fe640 1 -- 172.21.15.70:0/19381387 >> v1:172.21.15.70:6790/0 conn(0x7fb75013cbe0 legacy=0x7fb75014ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7fb74f7fe640 1 -- 172.21.15.70:0/19381387 >> v1:172.21.15.70:6789/0 conn(0x7fb750071bb0 legacy=0x7fb75013c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7fb74f7fe640 1 -- 172.21.15.70:0/19381387 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb750074db0 con 0x7fb7500710d0 2024-09-18T07:14:18.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7fb74f7fe640 1 -- 172.21.15.70:0/19381387 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb740001690 con 0x7fb7500710d0 2024-09-18T07:14:18.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f8d735b4640 1 -- start start 2024-09-18T07:14:18.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2016144428 0 0) 0x7f82480ac750 con 0x7f82480729a0 2024-09-18T07:14:18.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f822c003a10 con 0x7f82480729a0 2024-09-18T07:14:18.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3647641914 0 0) 0x7f82480ad790 con 0x7f8248073b60 2024-09-18T07:14:18.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f82480ac750 con 0x7f8248073b60 2024-09-18T07:14:18.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8238001770 con 0x7f8248073b60 2024-09-18T07:14:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f9bdcaa9640 1 Processor -- start 2024-09-18T07:14:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7fb74f7fe640 1 -- 172.21.15.70:0/19381387 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb7400018b0 con 0x7fb7500710d0 2024-09-18T07:14:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7fb74f7fe640 1 -- 172.21.15.70:0/19381387 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb740001cc0 con 0x7fb7500710d0 2024-09-18T07:14:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f94be7fc640 1 -- 172.21.15.70:0/3696179262 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c014e060 con 0x7f94c0074fb0 2024-09-18T07:14:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f94be7fc640 1 -- 172.21.15.70:0/3696179262 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f94b0001570 con 0x7f94c0074fb0 2024-09-18T07:14:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.695+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2964924505 0 0) 0x7f822c003520 con 0x7f82481452e0 2024-09-18T07:14:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.695+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f82480ad790 con 0x7f82481452e0 2024-09-18T07:14:18.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.695+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8244001920 con 0x7f82481452e0 2024-09-18T07:14:18.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.694+0000 7f9bdcaa9640 1 -- start start 2024-09-18T07:14:18.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7fb756d2c640 1 -- 172.21.15.70:0/19381387 >> v1:172.21.15.172:6789/0 conn(0x7fb7500710d0 legacy=0x7fb75013f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f8d70b4f640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f8d6c0af050 0x7f8d6c1476c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37942/0 (socket says 172.21.15.70:37942) 2024-09-18T07:14:18.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f8d70b4f640 1 -- 172.21.15.70:0/3939480705 learned_addr learned my addr 172.21.15.70:0/3939480705 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7fb756d2c640 1 -- 172.21.15.70:0/19381387 shutdown_connections 2024-09-18T07:14:18.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7fb756d2c640 1 -- 172.21.15.70:0/19381387 >> 172.21.15.70:0/19381387 conn(0x7fb7500b5160 msgr2=0x7fb7500b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:18.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f8d735b4640 1 -- 172.21.15.70:0/3939480705 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6c0ac220 con 0x7f8d6c0aacf0 2024-09-18T07:14:18.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f8d735b4640 1 -- 172.21.15.70:0/3939480705 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6c0ad290 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7fb756d2c640 1 -- 172.21.15.70:0/19381387 shutdown_connections 2024-09-18T07:14:18.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7fb756d2c640 1 -- 172.21.15.70:0/19381387 wait complete. 2024-09-18T07:14:18.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f9bdcaa9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bd8073df0 con 0x7f9bd80715e0 2024-09-18T07:14:18.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f9bdcaa9640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bd8071f50 con 0x7f9bd80719c0 2024-09-18T07:14:18.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f9bdcaa9640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bd813ffc0 con 0x7f9bd8074b00 2024-09-18T07:14:18.709 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f8d735b4640 1 -- 172.21.15.70:0/3939480705 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6c145150 con 0x7f8d6c0af050 2024-09-18T07:14:18.709 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.695+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2240813787 0 0) 0x7f82480ac750 con 0x7f8248073b60 2024-09-18T07:14:18.709 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.695+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 >> v1:172.21.15.70:6790/0 conn(0x7f82480729a0 legacy=0x7f82481408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.709 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.695+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 >> v1:172.21.15.70:6789/0 conn(0x7f82481452e0 legacy=0x7f8248201210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.695+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824813bb20 con 0x7f8248073b60 2024-09-18T07:14:18.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f9bd5d9b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9bd80719c0 0x7f9bd80743f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49878/0 (socket says 172.21.15.70:49878) 2024-09-18T07:14:18.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f9bd5d9b640 1 -- 172.21.15.70:0/4068111021 learned_addr learned my addr 172.21.15.70:0/4068111021 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f9bd559a640 1 -- 172.21.15.70:0/4068111021 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 578734516 0 0) 0x7f9bd8071f50 con 0x7f9bd80719c0 2024-09-18T07:14:18.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f9bd559a640 1 -- 172.21.15.70:0/4068111021 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bc0005730 con 0x7f9bd80719c0 2024-09-18T07:14:18.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f8d63fff640 1 -- 172.21.15.70:0/3939480705 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2606162198 0 0) 0x7f8d6c0ad290 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.696+0000 7f8d63fff640 1 -- 172.21.15.70:0/3939480705 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d58003520 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.695+0000 7f82506c6640 1 -- 172.21.15.70:0/2240583486 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f82480ac190 con 0x7f8248073b60 2024-09-18T07:14:18.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.697+0000 7f8d63fff640 1 -- 172.21.15.70:0/3939480705 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2216869637 0 0) 0x7f8d6c145150 con 0x7f8d6c0af050 2024-09-18T07:14:18.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.697+0000 7f8d63fff640 1 -- 172.21.15.70:0/3939480705 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d6c0ad290 con 0x7f8d6c0af050 2024-09-18T07:14:18.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.695+0000 7f82506c6640 1 -- 172.21.15.70:0/2240583486 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f8248202930 con 0x7f8248073b60 2024-09-18T07:14:18.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.697+0000 7f9bd559a640 1 -- 172.21.15.70:0/4068111021 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3470260656 0 0) 0x7f9bc0005730 con 0x7f9bd80719c0 2024-09-18T07:14:18.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.697+0000 7f9bd559a640 1 -- 172.21.15.70:0/4068111021 >> v1:172.21.15.70:6790/0 conn(0x7f9bd8074b00 legacy=0x7f9bd814b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-18T07:14:18.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.697+0000 7f8d63fff640 1 -- 172.21.15.70:0/3939480705 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 457944206 0 0) 0x7f8d58003520 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.697+0000 7f8d63fff640 1 -- 172.21.15.70:0/3939480705 >> v1:172.21.15.70:6790/0 conn(0x7f8d6c0af050 legacy=0x7f8d6c1476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.697+0000 7f8d63fff640 1 -- 172.21.15.70:0/3939480705 >> v1:172.21.15.70:6789/0 conn(0x7f8d6c0aacf0 legacy=0x7f8d6c144a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.697+0000 7f8d63fff640 1 -- 172.21.15.70:0/3939480705 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c147dd0 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.697+0000 7f8d63fff640 1 -- 172.21.15.70:0/3939480705 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8d5c001690 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.697+0000 7f94be7fc640 1 -- 172.21.15.70:0/3696179262 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f94b0001710 con 0x7f94c0074fb0 2024-09-18T07:14:18.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.697+0000 7f94be7fc640 1 -- 172.21.15.70:0/3696179262 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f94b0001ba0 con 0x7f94c0074fb0 2024-09-18T07:14:18.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.697+0000 7f9bd559a640 1 -- 172.21.15.70:0/4068111021 >> v1:172.21.15.70:6789/0 conn(0x7f9bd80715e0 legacy=0x7f9bd81405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.697+0000 7f9bd559a640 1 -- 172.21.15.70:0/4068111021 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd80b0130 con 0x7f9bd80719c0 2024-09-18T07:14:18.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7f9bd559a640 1 -- 172.21.15.70:0/4068111021 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f9bc8001a10 con 0x7f9bd80719c0 2024-09-18T07:14:18.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8238001bc0 con 0x7f8248073b60 2024-09-18T07:14:18.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7f94c6189640 1 -- 172.21.15.70:0/3696179262 >> v1:172.21.15.70:6789/0 conn(0x7f94c0074fb0 legacy=0x7f94c0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8238004980 con 0x7f8248073b60 2024-09-18T07:14:18.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7f82506c6640 1 -- 172.21.15.70:0/2240583486 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8248072100 con 0x7f8248073b60 2024-09-18T07:14:18.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7fb756d2c640 1 Processor -- start 2024-09-18T07:14:18.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7f8d63fff640 1 -- 172.21.15.70:0/3939480705 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8d5c001830 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7f8d63fff640 1 -- 172.21.15.70:0/3939480705 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8d5c001c80 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7f94c6189640 1 -- 172.21.15.70:0/3696179262 shutdown_connections 2024-09-18T07:14:18.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7f94c6189640 1 -- 172.21.15.70:0/3696179262 >> 172.21.15.70:0/3696179262 conn(0x7f94c006d180 msgr2=0x7f94c00b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:18.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7fb756d2c640 1 -- start start 2024-09-18T07:14:18.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7f9bd559a640 1 -- 172.21.15.70:0/4068111021 <== mon.1 v1:172.21.15.172:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9bc8001c20 con 0x7f9bd80719c0 2024-09-18T07:14:18.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7f9bd559a640 1 -- 172.21.15.70:0/4068111021 <== mon.1 v1:172.21.15.172:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f9bc80020b0 con 0x7f9bd80719c0 2024-09-18T07:14:18.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7f94c6189640 1 -- 172.21.15.70:0/3696179262 shutdown_connections 2024-09-18T07:14:18.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.698+0000 7f94c6189640 1 -- 172.21.15.70:0/3696179262 wait complete. 2024-09-18T07:14:18.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.699+0000 7f8d735b4640 1 -- 172.21.15.70:0/3939480705 >> v1:172.21.15.172:6789/0 conn(0x7f8d6c0ab0d0 legacy=0x7f8d6c152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.699+0000 7f8d735b4640 1 -- 172.21.15.70:0/3939480705 shutdown_connections 2024-09-18T07:14:18.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.699+0000 7f8d735b4640 1 -- 172.21.15.70:0/3939480705 >> 172.21.15.70:0/3939480705 conn(0x7f8d6c0b1b70 msgr2=0x7f8d6c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:18.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.699+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/4068111021 >> v1:172.21.15.172:6789/0 conn(0x7f9bd80719c0 legacy=0x7f9bd80743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.699+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/4068111021 shutdown_connections 2024-09-18T07:14:18.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.699+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/4068111021 >> 172.21.15.70:0/4068111021 conn(0x7f9bd806cf40 msgr2=0x7f9bd806d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:18.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.699+0000 7f1febb10640 1 Processor -- start 2024-09-18T07:14:18.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.699+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/4068111021 shutdown_connections 2024-09-18T07:14:18.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.700+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/4068111021 wait complete. 2024-09-18T07:14:18.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.700+0000 7f94c6189640 1 Processor -- start 2024-09-18T07:14:18.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.700+0000 7fb754ac8640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb7500710d0 0x7fb7501405f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52110/0 (socket says 172.21.15.70:52110) 2024-09-18T07:14:18.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.700+0000 7fb754ac8640 1 -- 172.21.15.70:0/1315695399 learned_addr learned my addr 172.21.15.70:0/1315695399 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.700+0000 7f1febb10640 1 -- start start 2024-09-18T07:14:18.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.700+0000 7fb756d2c640 1 -- 172.21.15.70:0/1315695399 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb750142b30 con 0x7fb7500710d0 2024-09-18T07:14:18.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.700+0000 7f94c6189640 1 -- start start 2024-09-18T07:14:18.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.700+0000 7f8d735b4640 1 -- 172.21.15.70:0/3939480705 shutdown_connections 2024-09-18T07:14:18.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.700+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f8238004b20 con 0x7f8248073b60 2024-09-18T07:14:18.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.701+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f8238095d40 con 0x7f8248073b60 2024-09-18T07:14:18.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.701+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3079982312 0 0) 0x7fb750142b30 con 0x7fb7500710d0 2024-09-18T07:14:18.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.701+0000 7fb756d2c640 1 -- 172.21.15.70:0/1315695399 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb750140ea0 con 0x7fb75013cbe0 2024-09-18T07:14:18.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.701+0000 7fb756d2c640 1 -- 172.21.15.70:0/1315695399 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb750142da0 con 0x7fb750071bb0 2024-09-18T07:14:18.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.700+0000 7f8d735b4640 1 -- 172.21.15.70:0/3939480705 wait complete. 2024-09-18T07:14:18.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.701+0000 7f1febb10640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe40ac220 con 0x7f1fe40ab0d0 2024-09-18T07:14:18.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.701+0000 7f1febb10640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe40ad290 con 0x7f1fe40af050 2024-09-18T07:14:18.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.701+0000 7f1febb10640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe4145150 con 0x7f1fe40aacf0 2024-09-18T07:14:18.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.701+0000 7f1fe98ac640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f1fe40aacf0 0x7f1fe4144a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37964/0 (socket says 172.21.15.70:37964) 2024-09-18T07:14:18.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.701+0000 7f1fe90ab640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1fe40af050 0x7f1fe41476c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49882/0 (socket says 172.21.15.70:49882) 2024-09-18T07:14:18.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.701+0000 7f1fe90ab640 1 -- 172.21.15.70:0/22906301 learned_addr learned my addr 172.21.15.70:0/22906301 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.701+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb73c003520 con 0x7fb7500710d0 2024-09-18T07:14:18.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f9bdcaa9640 1 Processor -- start 2024-09-18T07:14:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f1fe88aa640 1 -- 172.21.15.70:0/22906301 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2223406737 0 0) 0x7f1fe40ac220 con 0x7f1fe40ab0d0 2024-09-18T07:14:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f1fe88aa640 1 -- 172.21.15.70:0/22906301 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fcc003520 con 0x7f1fe40ab0d0 2024-09-18T07:14:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f1fe88aa640 1 -- 172.21.15.70:0/22906301 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2618611176 0 0) 0x7f1fe4145150 con 0x7f1fe40aacf0 2024-09-18T07:14:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3199918063 0 0) 0x7fb750142da0 con 0x7fb750071bb0 2024-09-18T07:14:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb750142b30 con 0x7fb750071bb0 2024-09-18T07:14:18.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f1fe88aa640 1 -- 172.21.15.70:0/22906301 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fe40ac220 con 0x7f1fe40aacf0 2024-09-18T07:14:18.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f1fe88aa640 1 -- 172.21.15.70:0/22906301 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3294891170 0 0) 0x7f1fe40ac220 con 0x7f1fe40aacf0 2024-09-18T07:14:18.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f1fe88aa640 1 -- 172.21.15.70:0/22906301 >> v1:172.21.15.172:6789/0 conn(0x7f1fe40af050 legacy=0x7f1fe41476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f1fe88aa640 1 -- 172.21.15.70:0/22906301 >> v1:172.21.15.70:6789/0 conn(0x7f1fe40ab0d0 legacy=0x7f1fe4152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f1fe88aa640 1 -- 172.21.15.70:0/22906301 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe4147dd0 con 0x7f1fe40aacf0 2024-09-18T07:14:18.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f1fe88aa640 1 -- 172.21.15.70:0/22906301 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f1fd40015a0 con 0x7f1fe40aacf0 2024-09-18T07:14:18.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 869119637 0 0) 0x7fb73c003520 con 0x7fb7500710d0 2024-09-18T07:14:18.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb750142da0 con 0x7fb7500710d0 2024-09-18T07:14:18.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3397027431 0 0) 0x7fb750142b30 con 0x7fb750071bb0 2024-09-18T07:14:18.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb73c003520 con 0x7fb750071bb0 2024-09-18T07:14:18.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb748001c90 con 0x7fb7500710d0 2024-09-18T07:14:18.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb740001c00 con 0x7fb750071bb0 2024-09-18T07:14:18.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f8d735b4640 1 Processor -- start 2024-09-18T07:14:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f1fe88aa640 1 -- 172.21.15.70:0/22906301 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1fd4001740 con 0x7f1fe40aacf0 2024-09-18T07:14:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f8d735b4640 1 -- start start 2024-09-18T07:14:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7f1fe88aa640 1 -- 172.21.15.70:0/22906301 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f1fd4001b50 con 0x7f1fe40aacf0 2024-09-18T07:14:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.702+0000 7f9bdcaa9640 1 -- start start 2024-09-18T07:14:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 569312202 0 0) 0x7fb750142da0 con 0x7fb7500710d0 2024-09-18T07:14:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 >> v1:172.21.15.70:6790/0 conn(0x7fb750071bb0 legacy=0x7fb750141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 >> v1:172.21.15.172:6789/0 conn(0x7fb75013cbe0 legacy=0x7fb7501d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7501d1a60 con 0x7fb7500710d0 2024-09-18T07:14:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7fb756d2c640 1 -- 172.21.15.70:0/1315695399 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb750141a60 con 0x7fb7500710d0 2024-09-18T07:14:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7fb756d2c640 1 -- 172.21.15.70:0/1315695399 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb7501d3f20 con 0x7fb7500710d0 2024-09-18T07:14:18.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f823805e6b0 con 0x7f8248073b60 2024-09-18T07:14:18.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7f9bd659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9bd80715e0 0x7f9bd813ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52136/0 (socket says 172.21.15.70:52136) 2024-09-18T07:14:18.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7f9bd5d9b640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f9bd80719c0 0x7f9bd813b3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37984/0 (socket says 172.21.15.70:37984) 2024-09-18T07:14:18.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7f9bd659c640 1 -- 172.21.15.70:0/888608879 learned_addr learned my addr 172.21.15.70:0/888608879 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7f1febb10640 1 -- 172.21.15.70:0/22906301 >> v1:172.21.15.70:6790/0 conn(0x7f1fe40aacf0 legacy=0x7f1fe4144a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7f1febb10640 1 -- 172.21.15.70:0/22906301 shutdown_connections 2024-09-18T07:14:18.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.703+0000 7f1febb10640 1 -- 172.21.15.70:0/22906301 >> 172.21.15.70:0/22906301 conn(0x7f1fe40b1b70 msgr2=0x7f1fe40b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:18.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.704+0000 7f8d735b4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6c14a760 con 0x7f8d6c0aacf0 2024-09-18T07:14:18.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.704+0000 7f8d735b4640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6c14b5a0 con 0x7f8d6c0af050 2024-09-18T07:14:18.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.704+0000 7f8d735b4640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6c14b880 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.704+0000 7f8d70b4f640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f8d6c0ab0d0 0x7f8d6c13e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:37992/0 (socket says 172.21.15.70:37992) 2024-09-18T07:14:18.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.704+0000 7f8d70b4f640 1 -- 172.21.15.70:0/107129838 learned_addr learned my addr 172.21.15.70:0/107129838 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.704+0000 7f1febb10640 1 -- 172.21.15.70:0/22906301 shutdown_connections 2024-09-18T07:14:18.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.704+0000 7f1febb10640 1 -- 172.21.15.70:0/22906301 wait complete. 2024-09-18T07:14:18.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.704+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 339690482 0 0) 0x7f8d6c14b880 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.704+0000 7fb756d2c640 1 -- 172.21.15.70:0/1315695399 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7500b4c70 con 0x7fb7500710d0 2024-09-18T07:14:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.704+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/888608879 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bd8142210 con 0x7f9bd80715e0 2024-09-18T07:14:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.705+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/888608879 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bd81424f0 con 0x7f9bd8074b00 2024-09-18T07:14:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.705+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/888608879 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bd8144520 con 0x7f9bd80719c0 2024-09-18T07:14:18.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.704+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d50003520 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.705+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3013935959 0 0) 0x7f8d6c14a760 con 0x7f8d6c0aacf0 2024-09-18T07:14:18.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.705+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d500048e0 con 0x7f8d6c0aacf0 2024-09-18T07:14:18.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.705+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1934887000 0 0) 0x7f8d6c14b5a0 con 0x7f8d6c0af050 2024-09-18T07:14:18.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.705+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d6c14a760 con 0x7f8d6c0af050 2024-09-18T07:14:18.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.705+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 82565050 0 0) 0x7f9bd8144520 con 0x7f9bd80719c0 2024-09-18T07:14:18.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.705+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bb8003520 con 0x7f9bd80719c0 2024-09-18T07:14:18.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.705+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 982774129 0 0) 0x7f8d50003520 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.705+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8d6c14b5a0 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1056130911 0 0) 0x7f9bb8003520 con 0x7f9bd80719c0 2024-09-18T07:14:18.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9bd8144520 con 0x7f9bd80719c0 2024-09-18T07:14:18.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f9bc80018d0 con 0x7f9bd80719c0 2024-09-18T07:14:18.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7f1febb10640 1 Processor -- start 2024-09-18T07:14:18.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7f1febb10640 1 -- start start 2024-09-18T07:14:18.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8d68001950 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 794731535 0 0) 0x7f8d6c14a760 con 0x7f8d6c0af050 2024-09-18T07:14:18.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8d50005420 con 0x7f8d6c0af050 2024-09-18T07:14:18.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8d5c001b80 con 0x7f8d6c0af050 2024-09-18T07:14:18.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb748004b60 con 0x7fb7500710d0 2024-09-18T07:14:18.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fb748004eb0 con 0x7fb7500710d0 2024-09-18T07:14:18.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fb74801dcc0 con 0x7fb7500710d0 2024-09-18T07:14:18.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2970810919 0 0) 0x7f9bd8144520 con 0x7f9bd80719c0 2024-09-18T07:14:18.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 >> v1:172.21.15.172:6789/0 conn(0x7f9bd8074b00 legacy=0x7f9bd81e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 >> v1:172.21.15.70:6789/0 conn(0x7f9bd80715e0 legacy=0x7f9bd813ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd8144140 con 0x7f9bd80719c0 2024-09-18T07:14:18.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/888608879 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd8141340 con 0x7f9bd80719c0 2024-09-18T07:14:18.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/888608879 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f9bd813f960 con 0x7f9bd80719c0 2024-09-18T07:14:18.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7fb748096430 con 0x7fb7500710d0 2024-09-18T07:14:18.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f1febb10640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe413eea0 con 0x7f1fe40ab0d0 2024-09-18T07:14:18.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f1febb10640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe413db70 con 0x7f1fe4141630 2024-09-18T07:14:18.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4099444474 0 0) 0x7f8d6c14b5a0 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.706+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 >> v1:172.21.15.172:6789/0 conn(0x7f8d6c0af050 legacy=0x7f8d6c1e7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 >> v1:172.21.15.70:6789/0 conn(0x7f8d6c0aacf0 legacy=0x7f8d6c14a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c14aa00 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f8d735b4640 1 -- 172.21.15.70:0/107129838 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6c147150 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f1febb10640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe414a160 con 0x7f1fe40af050 2024-09-18T07:14:18.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f1fea0ad640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1fe4141630 0x7f1fe41e89d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49928/0 (socket says 172.21.15.70:49928) 2024-09-18T07:14:18.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f1fea0ad640 1 -- 172.21.15.70:0/840413851 learned_addr learned my addr 172.21.15.70:0/840413851 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1370480378 0 0) 0x7f1fe413db70 con 0x7f1fe4141630 2024-09-18T07:14:18.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.707+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fd0003520 con 0x7f1fe4141630 2024-09-18T07:14:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.708+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1869539416 0 0) 0x7f1fd0003520 con 0x7f1fe4141630 2024-09-18T07:14:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.708+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1fe413db70 con 0x7f1fe4141630 2024-09-18T07:14:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.708+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f1fdc0019e0 con 0x7f1fe4141630 2024-09-18T07:14:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.708+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9bc8001c20 con 0x7f9bd80719c0 2024-09-18T07:14:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.708+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f9bc8005190 con 0x7f9bd80719c0 2024-09-18T07:14:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.705+0000 7f94c6189640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c00ac1a0 con 0x7f94c00abcf0 2024-09-18T07:14:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.705+0000 7f94c6189640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c00acf30 con 0x7f94c00b2a00 2024-09-18T07:14:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.705+0000 7f94c6189640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c00ad210 con 0x7f94c013d150 2024-09-18T07:14:18.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.709+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/888608879 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd80008d0 con 0x7f9bd80719c0 2024-09-18T07:14:18.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.709+0000 7f94bf7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f94c00b2a00 0x7f94c0139d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.70:49934/0 (socket says 172.21.15.70:49934) 2024-09-18T07:14:18.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.709+0000 7f94bf7fe640 1 -- 172.21.15.70:0/1431340281 learned_addr learned my addr 172.21.15.70:0/1431340281 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:18.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.708+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1865781154 0 0) 0x7f1fe413db70 con 0x7f1fe4141630 2024-09-18T07:14:18.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.708+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 >> v1:172.21.15.70:6790/0 conn(0x7f1fe40af050 legacy=0x7f1fe4149a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.708+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 >> v1:172.21.15.70:6789/0 conn(0x7f1fe40ab0d0 legacy=0x7f1fe4139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.708+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe414b190 con 0x7f1fe4141630 2024-09-18T07:14:18.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.708+0000 7f1febb10640 1 -- 172.21.15.70:0/840413851 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe4141ad0 con 0x7f1fe4141630 2024-09-18T07:14:18.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.708+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1fdc001c80 con 0x7f1fe4141630 2024-09-18T07:14:18.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.708+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f1fdc003a40 con 0x7f1fe4141630 2024-09-18T07:14:18.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.709+0000 7f1febb10640 1 -- 172.21.15.70:0/840413851 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f1fe4139d70 con 0x7f1fe4141630 2024-09-18T07:14:18.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.710+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2499855793 0 0) 0x7f94c00acf30 con 0x7f94c00b2a00 2024-09-18T07:14:18.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.710+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94ac003520 con 0x7f94c00b2a00 2024-09-18T07:14:18.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.710+0000 7f8d735b4640 1 -- 172.21.15.70:0/107129838 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f8d6c13eb50 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.710+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3021732757 0 0) 0x7f94ac003520 con 0x7f94c00b2a00 2024-09-18T07:14:18.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.710+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f94ac0039b0 con 0x7f94c00b2a00 2024-09-18T07:14:18.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.710+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f94b4001990 con 0x7f94c00b2a00 2024-09-18T07:14:18.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.711+0000 7f8d735b4640 1 -- 172.21.15.70:0/107129838 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d6c0008d0 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.711+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 652638215 0 0) 0x7f94c00ad210 con 0x7f94c013d150 2024-09-18T07:14:18.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.711+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94ac003520 con 0x7f94c013d150 2024-09-18T07:14:18.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.711+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 <== mon.1 v1:172.21.15.172:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1944196975 0 0) 0x7f94ac0039b0 con 0x7f94c00b2a00 2024-09-18T07:14:18.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.711+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 >> v1:172.21.15.70:6790/0 conn(0x7f94c013d150 legacy=0x7f94c00ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.711+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 >> v1:172.21.15.70:6789/0 conn(0x7f94c00abcf0 legacy=0x7f94c01defd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:18.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.711+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c00aaf20 con 0x7f94c00b2a00 2024-09-18T07:14:18.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.711+0000 7f94c6189640 1 -- 172.21.15.70:0/1431340281 --> v1:172.21.15.172:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f94c00aab40 con 0x7f94c00b2a00 2024-09-18T07:14:18.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.711+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 <== mon.1 v1:172.21.15.172:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f94b40047e0 con 0x7f94c00b2a00 2024-09-18T07:14:18.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.711+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 <== mon.1 v1:172.21.15.172:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f94b4004bf0 con 0x7f94c00b2a00 2024-09-18T07:14:18.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.710+0000 7f1febb10640 1 -- 172.21.15.70:0/840413851 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe40b3dd0 con 0x7f1fe4141630 2024-09-18T07:14:18.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.712+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8d680047e0 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.712+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f8d680017b0 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.712+0000 7f94c6189640 1 -- 172.21.15.70:0/1431340281 --> v1:172.21.15.172:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f94c00ae640 con 0x7f94c00b2a00 2024-09-18T07:14:18.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.713+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f9bc801df60 con 0x7f9bd80719c0 2024-09-18T07:14:18.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.713+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7fb74805ee20 con 0x7fb7500710d0 2024-09-18T07:14:18.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.714+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f94b4004d90 con 0x7f94c00b2a00 2024-09-18T07:14:18.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.714+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f8d6801d680 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.714+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 <== mon.1 v1:172.21.15.172:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f1fdc003be0 con 0x7f1fe4141630 2024-09-18T07:14:18.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.714+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f1fdc0960e0 con 0x7f1fe4141630 2024-09-18T07:14:18.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.715+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f9bc80963b0 con 0x7f9bd80719c0 2024-09-18T07:14:18.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.715+0000 7f94c6189640 1 -- 172.21.15.70:0/1431340281 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c00008d0 con 0x7f94c00b2a00 2024-09-18T07:14:18.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.716+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 <== mon.1 v1:172.21.15.172:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f94b4094930 con 0x7f94c00b2a00 2024-09-18T07:14:18.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.716+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f1fdc05ea50 con 0x7f1fe4141630 2024-09-18T07:14:18.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.721+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 <== mon.1 v1:172.21.15.172:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f94b405e6b0 con 0x7f94c00b2a00 2024-09-18T07:14:18.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.723+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f8d680948e0 con 0x7f8d6c0ab0d0 2024-09-18T07:14:18.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.722+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f9bc805ed20 con 0x7f9bd80719c0 2024-09-18T07:14:18.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:18.730+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f8d6805e660 con 0x7f8d6c0ab0d0 2024-09-18T07:14:19.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.055+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2308365993 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc650005370 con 0x7fc6600067e0 2024-09-18T07:14:19.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.055+0000 7fc65e7fc640 1 -- 172.21.15.70:0/2308365993 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (unknown 1482059429 0 3056272221) 0x7fc66405e040 con 0x7fc6600067e0 2024-09-18T07:14:19.058 INFO:teuthology.orchestra.run.smithi070.stdout:55834574852 2024-09-18T07:14:19.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.059+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2308365993 >> v1:172.21.15.70:6816/1657990791 conn(0x7fc64c0795d0 legacy=0x7fc64c07ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.060+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2308365993 >> v1:172.21.15.172:6789/0 conn(0x7fc6600067e0 legacy=0x7fc66002ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.060+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2308365993 shutdown_connections 2024-09-18T07:14:19.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.060+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2308365993 >> 172.21.15.70:0/2308365993 conn(0x7fc660005840 msgr2=0x7fc660005010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:19.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.060+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2308365993 shutdown_connections 2024-09-18T07:14:19.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.060+0000 7fc66fc2a640 1 -- 172.21.15.70:0/2308365993 wait complete. 2024-09-18T07:14:19.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.060+0000 7f766a0ca640 1 -- 172.21.15.70:0/3894634665 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7644005370 con 0x7f765c0306d0 2024-09-18T07:14:19.067 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-18T07:14:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.061+0000 7f7659ffb640 1 -- 172.21.15.70:0/3894634665 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (unknown 92182286 0 3887400221) 0x7f766005d5f0 con 0x7f765c0306d0 2024-09-18T07:14:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.065+0000 7f766a0ca640 1 -- 172.21.15.70:0/3894634665 >> v1:172.21.15.70:6816/1657990791 conn(0x7f764807dbe0 legacy=0x7f76480800a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.065+0000 7f766a0ca640 1 -- 172.21.15.70:0/3894634665 >> v1:172.21.15.70:6789/0 conn(0x7f765c0306d0 legacy=0x7f765c0b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.065+0000 7f766a0ca640 1 -- 172.21.15.70:0/3894634665 shutdown_connections 2024-09-18T07:14:19.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.065+0000 7f766a0ca640 1 -- 172.21.15.70:0/3894634665 >> 172.21.15.70:0/3894634665 conn(0x7f765c005460 msgr2=0x7f765c032ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:19.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.065+0000 7f766a0ca640 1 -- 172.21.15.70:0/3894634665 shutdown_connections 2024-09-18T07:14:19.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.065+0000 7f766a0ca640 1 -- 172.21.15.70:0/3894634665 wait complete. 2024-09-18T07:14:19.080 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.7 2024-09-18T07:14:19.080 DEBUG:teuthology.parallel:result is None 2024-09-18T07:14:19.083 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.2 2024-09-18T07:14:19.083 DEBUG:teuthology.parallel:result is None 2024-09-18T07:14:19.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.100+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/888608879 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f9bd8144310 con 0x7f9bd80719c0 2024-09-18T07:14:19.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.101+0000 7f9bc6ffd640 1 -- 172.21.15.70:0/888608879 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (unknown 1823589948 0 3887400221) 0x7f9bc805e620 con 0x7f9bd80719c0 2024-09-18T07:14:19.104 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-18T07:14:19.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.103+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/888608879 >> v1:172.21.15.70:6816/1657990791 conn(0x7f9bb80797f0 legacy=0x7f9bb807bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.103+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/888608879 >> v1:172.21.15.70:6790/0 conn(0x7f9bd80719c0 legacy=0x7f9bd813b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.103+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/888608879 shutdown_connections 2024-09-18T07:14:19.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.103+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/888608879 >> 172.21.15.70:0/888608879 conn(0x7f9bd806cf40 msgr2=0x7f9bd80b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:19.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.103+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/888608879 shutdown_connections 2024-09-18T07:14:19.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.103+0000 7f9bdcaa9640 1 -- 172.21.15.70:0/888608879 wait complete. 2024-09-18T07:14:19.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.107+0000 7f8d735b4640 1 -- 172.21.15.70:0/107129838 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8d6c13eb50 con 0x7f8d6c0ab0d0 2024-09-18T07:14:19.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.108+0000 7f8d61ffb640 1 -- 172.21.15.70:0/107129838 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (unknown 383520633 0 3887400221) 0x7f8d68062a40 con 0x7f8d6c0ab0d0 2024-09-18T07:14:19.110 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-18T07:14:19.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.110+0000 7f8d735b4640 1 -- 172.21.15.70:0/107129838 >> v1:172.21.15.70:6816/1657990791 conn(0x7f8d5007a0c0 legacy=0x7f8d5007c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.110+0000 7f8d735b4640 1 -- 172.21.15.70:0/107129838 >> v1:172.21.15.70:6790/0 conn(0x7f8d6c0ab0d0 legacy=0x7f8d6c13e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.110+0000 7f8d735b4640 1 -- 172.21.15.70:0/107129838 shutdown_connections 2024-09-18T07:14:19.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.110+0000 7f8d735b4640 1 -- 172.21.15.70:0/107129838 >> 172.21.15.70:0/107129838 conn(0x7f8d6c0b1b70 msgr2=0x7f8d6c069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:19.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.110+0000 7f8d735b4640 1 -- 172.21.15.70:0/107129838 shutdown_connections 2024-09-18T07:14:19.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.110+0000 7f8d735b4640 1 -- 172.21.15.70:0/107129838 wait complete. 2024-09-18T07:14:19.117 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.4 2024-09-18T07:14:19.117 DEBUG:teuthology.parallel:result is None 2024-09-18T07:14:19.124 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.3 2024-09-18T07:14:19.124 DEBUG:teuthology.parallel:result is None 2024-09-18T07:14:19.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.127+0000 7f82506c6640 1 -- 172.21.15.70:0/2240583486 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f82480ac9e0 con 0x7f8248073b60 2024-09-18T07:14:19.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.127+0000 7f823effd640 1 -- 172.21.15.70:0/2240583486 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (unknown 574334944 0 3887400221) 0x7f823805dfb0 con 0x7f8248073b60 2024-09-18T07:14:19.130 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-18T07:14:19.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.129+0000 7f82506c6640 1 -- 172.21.15.70:0/2240583486 >> v1:172.21.15.70:6816/1657990791 conn(0x7f822c079b90 legacy=0x7f822c07c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.129+0000 7f82506c6640 1 -- 172.21.15.70:0/2240583486 >> v1:172.21.15.172:6789/0 conn(0x7f8248073b60 legacy=0x7f8248140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.129+0000 7f82506c6640 1 -- 172.21.15.70:0/2240583486 shutdown_connections 2024-09-18T07:14:19.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.129+0000 7f82506c6640 1 -- 172.21.15.70:0/2240583486 >> 172.21.15.70:0/2240583486 conn(0x7f82480b3c90 msgr2=0x7f824813e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:19.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.130+0000 7f82506c6640 1 -- 172.21.15.70:0/2240583486 shutdown_connections 2024-09-18T07:14:19.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.130+0000 7f82506c6640 1 -- 172.21.15.70:0/2240583486 wait complete. 2024-09-18T07:14:19.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.131+0000 7fb756d2c640 1 -- 172.21.15.70:0/1315695399 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb75006cfd0 con 0x7fb7500710d0 2024-09-18T07:14:19.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.131+0000 7fb74d7fa640 1 -- 172.21.15.70:0/1315695399 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (unknown 2131975755 0 3056272221) 0x7fb74805e720 con 0x7fb7500710d0 2024-09-18T07:14:19.134 INFO:teuthology.orchestra.run.smithi070.stdout:55834574852 2024-09-18T07:14:19.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.134+0000 7fb756d2c640 1 -- 172.21.15.70:0/1315695399 >> v1:172.21.15.70:6816/1657990791 conn(0x7fb73c0798d0 legacy=0x7fb73c07bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.134+0000 7fb756d2c640 1 -- 172.21.15.70:0/1315695399 >> v1:172.21.15.70:6789/0 conn(0x7fb7500710d0 legacy=0x7fb7501405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.134+0000 7f1febb10640 1 -- 172.21.15.70:0/840413851 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1fe40b2230 con 0x7f1fe4141630 2024-09-18T07:14:19.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.134+0000 7fb756d2c640 1 -- 172.21.15.70:0/1315695399 shutdown_connections 2024-09-18T07:14:19.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.134+0000 7fb756d2c640 1 -- 172.21.15.70:0/1315695399 >> 172.21.15.70:0/1315695399 conn(0x7fb7500b5160 msgr2=0x7fb7501397c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:19.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.134+0000 7fb756d2c640 1 -- 172.21.15.70:0/1315695399 shutdown_connections 2024-09-18T07:14:19.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.134+0000 7fb756d2c640 1 -- 172.21.15.70:0/1315695399 wait complete. 2024-09-18T07:14:19.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.134+0000 7f1fda7fc640 1 -- 172.21.15.70:0/840413851 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (unknown 1274345170 0 3056272221) 0x7f1fdc067020 con 0x7f1fe4141630 2024-09-18T07:14:19.138 INFO:teuthology.orchestra.run.smithi070.stdout:55834574852 2024-09-18T07:14:19.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.136+0000 7f94c6189640 1 -- 172.21.15.70:0/1431340281 --> v1:172.21.15.172:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f94c00aad10 con 0x7f94c00b2a00 2024-09-18T07:14:19.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.137+0000 7f1febb10640 1 -- 172.21.15.70:0/840413851 >> v1:172.21.15.70:6816/1657990791 conn(0x7f1fd00796f0 legacy=0x7f1fd007bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.137+0000 7f1febb10640 1 -- 172.21.15.70:0/840413851 >> v1:172.21.15.172:6789/0 conn(0x7f1fe4141630 legacy=0x7f1fe41e89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.137+0000 7f1febb10640 1 -- 172.21.15.70:0/840413851 shutdown_connections 2024-09-18T07:14:19.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.137+0000 7f1febb10640 1 -- 172.21.15.70:0/840413851 >> 172.21.15.70:0/840413851 conn(0x7f1fe40b1b70 msgr2=0x7f1fe40ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:19.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.137+0000 7f1febb10640 1 -- 172.21.15.70:0/840413851 shutdown_connections 2024-09-18T07:14:19.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.137+0000 7f1febb10640 1 -- 172.21.15.70:0/840413851 wait complete. 2024-09-18T07:14:19.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.137+0000 7f94a7fff640 1 -- 172.21.15.70:0/1431340281 <== mon.1 v1:172.21.15.172:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (unknown 832126871 0 3887400221) 0x7f94b405e6b0 con 0x7f94c00b2a00 2024-09-18T07:14:19.141 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-18T07:14:19.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.140+0000 7f94c6189640 1 -- 172.21.15.70:0/1431340281 >> v1:172.21.15.70:6816/1657990791 conn(0x7f94ac079ec0 legacy=0x7f94ac07c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.140+0000 7f94c6189640 1 -- 172.21.15.70:0/1431340281 >> v1:172.21.15.172:6789/0 conn(0x7f94c00b2a00 legacy=0x7f94c0139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.140+0000 7f94c6189640 1 -- 172.21.15.70:0/1431340281 shutdown_connections 2024-09-18T07:14:19.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.140+0000 7f94c6189640 1 -- 172.21.15.70:0/1431340281 >> 172.21.15.70:0/1431340281 conn(0x7f94c006d180 msgr2=0x7f94c0072080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:19.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.140+0000 7f94c6189640 1 -- 172.21.15.70:0/1431340281 shutdown_connections 2024-09-18T07:14:19.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.140+0000 7f94c6189640 1 -- 172.21.15.70:0/1431340281 wait complete. 2024-09-18T07:14:19.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.0 2024-09-18T07:14:19.146 DEBUG:teuthology.parallel:result is None 2024-09-18T07:14:19.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2024-09-18T07:14:19.150 DEBUG:teuthology.parallel:result is None 2024-09-18T07:14:19.154 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.6 2024-09-18T07:14:19.154 DEBUG:teuthology.parallel:result is None 2024-09-18T07:14:19.154 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.1 2024-09-18T07:14:19.155 DEBUG:teuthology.parallel:result is None 2024-09-18T07:14:19.155 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T07:14:19.155 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T07:14:19.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.291+0000 7fe1d77c2640 1 Processor -- start 2024-09-18T07:14:19.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.291+0000 7fe1d77c2640 1 -- start start 2024-09-18T07:14:19.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.291+0000 7fe1d77c2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1d00b1410 con 0x7fe1d00ae7f0 2024-09-18T07:14:19.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.291+0000 7fe1d77c2640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1d00acaa0 con 0x7fe1d0149910 2024-09-18T07:14:19.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1d77c2640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1d013d8c0 con 0x7fe1d00afce0 2024-09-18T07:14:19.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1d555e640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fe1d00afce0 0x7fe1d013d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:38022/0 (socket says 172.21.15.70:38022) 2024-09-18T07:14:19.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1d555e640 1 -- 172.21.15.70:0/241921069 learned_addr learned my addr 172.21.15.70:0/241921069 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:19.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1c7fff640 1 -- 172.21.15.70:0/241921069 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1468818398 0 0) 0x7fe1d013d8c0 con 0x7fe1d00afce0 2024-09-18T07:14:19.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1c7fff640 1 -- 172.21.15.70:0/241921069 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1bc003520 con 0x7fe1d00afce0 2024-09-18T07:14:19.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1c7fff640 1 -- 172.21.15.70:0/241921069 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1378493574 0 0) 0x7fe1d00b1410 con 0x7fe1d00ae7f0 2024-09-18T07:14:19.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1c7fff640 1 -- 172.21.15.70:0/241921069 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1d013d8c0 con 0x7fe1d00ae7f0 2024-09-18T07:14:19.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1c7fff640 1 -- 172.21.15.70:0/241921069 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4248892739 0 0) 0x7fe1bc003520 con 0x7fe1d00afce0 2024-09-18T07:14:19.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1c7fff640 1 -- 172.21.15.70:0/241921069 >> v1:172.21.15.172:6789/0 conn(0x7fe1d0149910 legacy=0x7fe1d014bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1c7fff640 1 -- 172.21.15.70:0/241921069 >> v1:172.21.15.70:6789/0 conn(0x7fe1d00ae7f0 legacy=0x7fe1d00abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1c7fff640 1 -- 172.21.15.70:0/241921069 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d0074b40 con 0x7fe1d00afce0 2024-09-18T07:14:19.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1c7fff640 1 -- 172.21.15.70:0/241921069 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fe1c0001690 con 0x7fe1d00afce0 2024-09-18T07:14:19.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1c7fff640 1 -- 172.21.15.70:0/241921069 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe1c00018b0 con 0x7fe1d00afce0 2024-09-18T07:14:19.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.292+0000 7fe1c7fff640 1 -- 172.21.15.70:0/241921069 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fe1c0001cc0 con 0x7fe1d00afce0 2024-09-18T07:14:19.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.293+0000 7fe1d77c2640 1 -- 172.21.15.70:0/241921069 >> v1:172.21.15.70:6790/0 conn(0x7fe1d00afce0 legacy=0x7fe1d013d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.293+0000 7fe1d77c2640 1 -- 172.21.15.70:0/241921069 shutdown_connections 2024-09-18T07:14:19.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.293+0000 7fe1d77c2640 1 -- 172.21.15.70:0/241921069 >> 172.21.15.70:0/241921069 conn(0x7fe1d00b1b70 msgr2=0x7fe1d00b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:19.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.293+0000 7fe1d77c2640 1 -- 172.21.15.70:0/241921069 shutdown_connections 2024-09-18T07:14:19.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.293+0000 7fe1d77c2640 1 -- 172.21.15.70:0/241921069 wait complete. 2024-09-18T07:14:19.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.294+0000 7fe1d77c2640 1 Processor -- start 2024-09-18T07:14:19.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.294+0000 7fe1d77c2640 1 -- start start 2024-09-18T07:14:19.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.294+0000 7fe1d77c2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1d013f1d0 con 0x7fe1d0149910 2024-09-18T07:14:19.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.294+0000 7fe1d77c2640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1d013f4b0 con 0x7fe1d00afce0 2024-09-18T07:14:19.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.294+0000 7fe1d77c2640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1d013f790 con 0x7fe1d00ae7f0 2024-09-18T07:14:19.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.294+0000 7fe1d555e640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fe1d00ae7f0 0x7fe1d0133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:38034/0 (socket says 172.21.15.70:38034) 2024-09-18T07:14:19.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.294+0000 7fe1d555e640 1 -- 172.21.15.70:0/3193445557 learned_addr learned my addr 172.21.15.70:0/3193445557 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:19.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4288146547 0 0) 0x7fe1d013f1d0 con 0x7fe1d0149910 2024-09-18T07:14:19.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1b4003520 con 0x7fe1d0149910 2024-09-18T07:14:19.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3717200284 0 0) 0x7fe1d013f790 con 0x7fe1d00ae7f0 2024-09-18T07:14:19.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1d013f1d0 con 0x7fe1d00ae7f0 2024-09-18T07:14:19.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3744180345 0 0) 0x7fe1b4003520 con 0x7fe1d0149910 2024-09-18T07:14:19.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe1d013f790 con 0x7fe1d0149910 2024-09-18T07:14:19.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fe1c8001b50 con 0x7fe1d0149910 2024-09-18T07:14:19.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2533017691 0 0) 0x7fe1d013f4b0 con 0x7fe1d00afce0 2024-09-18T07:14:19.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1b4003520 con 0x7fe1d00afce0 2024-09-18T07:14:19.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 909427658 0 0) 0x7fe1d013f1d0 con 0x7fe1d00ae7f0 2024-09-18T07:14:19.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe1d013f4b0 con 0x7fe1d00ae7f0 2024-09-18T07:14:19.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fe1c0001c00 con 0x7fe1d00ae7f0 2024-09-18T07:14:19.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4294961870 0 0) 0x7fe1d013f790 con 0x7fe1d0149910 2024-09-18T07:14:19.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 >> v1:172.21.15.70:6790/0 conn(0x7fe1d00ae7f0 legacy=0x7fe1d0133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 >> v1:172.21.15.172:6789/0 conn(0x7fe1d00afce0 legacy=0x7fe1d013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d013de00 con 0x7fe1d0149910 2024-09-18T07:14:19.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1d77c2640 1 -- 172.21.15.70:0/3193445557 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d013e320 con 0x7fe1d0149910 2024-09-18T07:14:19.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe1c8003970 con 0x7fe1d0149910 2024-09-18T07:14:19.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fe1c8003cc0 con 0x7fe1d0149910 2024-09-18T07:14:19.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.295+0000 7fe1d77c2640 1 -- 172.21.15.70:0/3193445557 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe1d01402c0 con 0x7fe1d0149910 2024-09-18T07:14:19.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.296+0000 7fe1d77c2640 1 -- 172.21.15.70:0/3193445557 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d0140650 con 0x7fe1d0149910 2024-09-18T07:14:19.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.297+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fe1c801ca90 con 0x7fe1d0149910 2024-09-18T07:14:19.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.297+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7fe1c80951c0 con 0x7fe1d0149910 2024-09-18T07:14:19.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.301+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7fe1c805db30 con 0x7fe1d0149910 2024-09-18T07:14:19.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.475+0000 7fe1d77c2640 1 -- 172.21.15.70:0/3193445557 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1d006cc40 con 0x7fe1b4079880 2024-09-18T07:14:19.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.476+0000 7fe1c5ffb640 1 -- 172.21.15.70:0/3193445557 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+35848 (unknown 2965378022 0 4039020994) 0x7fe1d006cc40 con 0x7fe1b4079880 2024-09-18T07:14:19.479 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:14:19.479 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-18T07:14:19.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.478+0000 7fe1d77c2640 1 -- 172.21.15.70:0/3193445557 >> v1:172.21.15.70:6816/1657990791 conn(0x7fe1b4079880 legacy=0x7fe1b407bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.478+0000 7fe1d77c2640 1 -- 172.21.15.70:0/3193445557 >> v1:172.21.15.70:6789/0 conn(0x7fe1d0149910 legacy=0x7fe1d01e2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.478+0000 7fe1d77c2640 1 -- 172.21.15.70:0/3193445557 shutdown_connections 2024-09-18T07:14:19.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.478+0000 7fe1d77c2640 1 -- 172.21.15.70:0/3193445557 >> 172.21.15.70:0/3193445557 conn(0x7fe1d00b1b70 msgr2=0x7fe1d00aa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:19.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.478+0000 7fe1d77c2640 1 -- 172.21.15.70:0/3193445557 shutdown_connections 2024-09-18T07:14:19.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.478+0000 7fe1d77c2640 1 -- 172.21.15.70:0/3193445557 wait complete. 2024-09-18T07:14:19.492 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-18T07:14:19.446578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":227064,"kb_used_data":2476,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754747656,"statfs":{"total":773094113280,"available":772861599744,"internally_reserved":0,"allocated":2535424,"data_stored":1553494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.206443"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.808451+0000","last_change":"2024-09-18T07:14:15.808642+0000","last_active":"2024-09-18T07:14:15.808451+0000","last_peered":"2024-09-18T07:14:15.808451+0000","last_clean":"2024-09-18T07:14:15.808451+0000","last_became_active":"2024-09-18T07:14:13.251609+0000","last_became_peered":"2024-09-18T07:14:13.251609+0000","last_unstale":"2024-09-18T07:14:15.808451+0000","last_undegraded":"2024-09-18T07:14:15.808451+0000","last_fullsized":"2024-09-18T07:14:15.808451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:01.748551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040903899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.249663+0000","last_change":"2024-09-18T07:14:15.249860+0000","last_active":"2024-09-18T07:14:15.249663+0000","last_peered":"2024-09-18T07:14:15.249663+0000","last_clean":"2024-09-18T07:14:15.249663+0000","last_became_active":"2024-09-18T07:14:13.251585+0000","last_became_peered":"2024-09-18T07:14:13.251585+0000","last_unstale":"2024-09-18T07:14:15.249663+0000","last_undegraded":"2024-09-18T07:14:15.249663+0000","last_fullsized":"2024-09-18T07:14:15.249663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:02:29.914389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039885799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.602830+0000","last_change":"2024-09-18T07:14:15.603034+0000","last_active":"2024-09-18T07:14:15.602830+0000","last_peered":"2024-09-18T07:14:15.602830+0000","last_clean":"2024-09-18T07:14:15.602830+0000","last_became_active":"2024-09-18T07:14:13.957168+0000","last_became_peered":"2024-09-18T07:14:13.957168+0000","last_unstale":"2024-09-18T07:14:15.602830+0000","last_undegraded":"2024-09-18T07:14:15.602830+0000","last_fullsized":"2024-09-18T07:14:15.602830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:57:48.846502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048630800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.249617+0000","last_change":"2024-09-18T07:14:15.249863+0000","last_active":"2024-09-18T07:14:15.249617+0000","last_peered":"2024-09-18T07:14:15.249617+0000","last_clean":"2024-09-18T07:14:15.249617+0000","last_became_active":"2024-09-18T07:14:13.956929+0000","last_became_peered":"2024-09-18T07:14:13.956929+0000","last_unstale":"2024-09-18T07:14:15.249617+0000","last_undegraded":"2024-09-18T07:14:15.249617+0000","last_fullsized":"2024-09-18T07:14:15.249617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:11:55.607274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045339399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.255874+0000","last_change":"2024-09-18T07:14:15.255874+0000","last_active":"2024-09-18T07:14:15.255874+0000","last_peered":"2024-09-18T07:14:15.255874+0000","last_clean":"2024-09-18T07:14:15.255874+0000","last_became_active":"2024-09-18T07:14:13.243579+0000","last_became_peered":"2024-09-18T07:14:13.243579+0000","last_unstale":"2024-09-18T07:14:15.255874+0000","last_undegraded":"2024-09-18T07:14:15.255874+0000","last_fullsized":"2024-09-18T07:14:15.255874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:28.722433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096575000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.398003+0000","last_change":"2024-09-18T07:14:15.398157+0000","last_active":"2024-09-18T07:14:15.398003+0000","last_peered":"2024-09-18T07:14:15.398003+0000","last_clean":"2024-09-18T07:14:15.398003+0000","last_became_active":"2024-09-18T07:14:13.392213+0000","last_became_peered":"2024-09-18T07:14:13.392213+0000","last_unstale":"2024-09-18T07:14:15.398003+0000","last_undegraded":"2024-09-18T07:14:15.398003+0000","last_fullsized":"2024-09-18T07:14:15.398003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:47:04.187501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000312059,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.602800+0000","last_change":"2024-09-18T07:14:15.603018+0000","last_active":"2024-09-18T07:14:15.602800+0000","last_peered":"2024-09-18T07:14:15.602800+0000","last_clean":"2024-09-18T07:14:15.602800+0000","last_became_active":"2024-09-18T07:14:13.251213+0000","last_became_peered":"2024-09-18T07:14:13.251213+0000","last_unstale":"2024-09-18T07:14:15.602800+0000","last_undegraded":"2024-09-18T07:14:15.602800+0000","last_fullsized":"2024-09-18T07:14:15.602800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:43:22.952115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046971900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.255106+0000","last_change":"2024-09-18T07:14:15.255301+0000","last_active":"2024-09-18T07:14:15.255106+0000","last_peered":"2024-09-18T07:14:15.255106+0000","last_clean":"2024-09-18T07:14:15.255106+0000","last_became_active":"2024-09-18T07:14:13.243793+0000","last_became_peered":"2024-09-18T07:14:13.243793+0000","last_unstale":"2024-09-18T07:14:15.255106+0000","last_undegraded":"2024-09-18T07:14:15.255106+0000","last_fullsized":"2024-09-18T07:14:15.255106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:24:14.077910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039863000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.602478+0000","last_change":"2024-09-18T07:14:06.881800+0000","last_active":"2024-09-18T07:14:15.602478+0000","last_peered":"2024-09-18T07:14:15.602478+0000","last_clean":"2024-09-18T07:14:15.602478+0000","last_became_active":"2024-09-18T07:14:06.880771+0000","last_became_peered":"2024-09-18T07:14:06.880771+0000","last_unstale":"2024-09-18T07:14:15.602478+0000","last_undegraded":"2024-09-18T07:14:15.602478+0000","last_fullsized":"2024-09-18T07:14:15.602478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:05.472923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:05.472923+0000","last_clean_scrub_stamp":"2024-09-18T07:14:05.472923+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:07.064297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":716800,"data_stored":591467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":57827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":62539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":62539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":184320,"data_stored":62539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":188416,"data_stored":62558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":716800,"data_stored":591467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":188416,"data_stored":62558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T07:14:19.492 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T07:14:19.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.587+0000 7f2a49132640 1 Processor -- start 2024-09-18T07:14:19.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.587+0000 7f2a49132640 1 -- start start 2024-09-18T07:14:19.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a49132640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a440b1410 con 0x7f2a440ae7f0 2024-09-18T07:14:19.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a49132640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a440acaa0 con 0x7f2a44149910 2024-09-18T07:14:19.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a49132640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a4413d8c0 con 0x7f2a440afce0 2024-09-18T07:14:19.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a42d9d640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f2a440afce0 0x7f2a4413d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:38036/0 (socket says 172.21.15.70:38036) 2024-09-18T07:14:19.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a42d9d640 1 -- 172.21.15.70:0/1202730278 learned_addr learned my addr 172.21.15.70:0/1202730278 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:19.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a3bfff640 1 --1- 172.21.15.70:0/1202730278 >> v1:172.21.15.70:6789/0 conn(0x7f2a440ae7f0 0x7f2a440abcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52198/0 (socket says 172.21.15.70:52198) 2024-09-18T07:14:19.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a4259c640 1 -- 172.21.15.70:0/1202730278 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2592139733 0 0) 0x7f2a440b1410 con 0x7f2a440ae7f0 2024-09-18T07:14:19.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a4259c640 1 -- 172.21.15.70:0/1202730278 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a30003520 con 0x7f2a440ae7f0 2024-09-18T07:14:19.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a4259c640 1 -- 172.21.15.70:0/1202730278 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2356853532 0 0) 0x7f2a4413d8c0 con 0x7f2a440afce0 2024-09-18T07:14:19.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a4259c640 1 -- 172.21.15.70:0/1202730278 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a440b1410 con 0x7f2a440afce0 2024-09-18T07:14:19.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a4259c640 1 -- 172.21.15.70:0/1202730278 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4250977666 0 0) 0x7f2a440b1410 con 0x7f2a440afce0 2024-09-18T07:14:19.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a4259c640 1 -- 172.21.15.70:0/1202730278 >> v1:172.21.15.172:6789/0 conn(0x7f2a44149910 legacy=0x7f2a4414bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a4259c640 1 -- 172.21.15.70:0/1202730278 >> v1:172.21.15.70:6789/0 conn(0x7f2a440ae7f0 legacy=0x7f2a440abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a4259c640 1 -- 172.21.15.70:0/1202730278 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a44074b40 con 0x7f2a440afce0 2024-09-18T07:14:19.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a4259c640 1 -- 172.21.15.70:0/1202730278 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2a34001670 con 0x7f2a440afce0 2024-09-18T07:14:19.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a4259c640 1 -- 172.21.15.70:0/1202730278 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2a34001810 con 0x7f2a440afce0 2024-09-18T07:14:19.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.588+0000 7f2a4259c640 1 -- 172.21.15.70:0/1202730278 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2a34001c20 con 0x7f2a440afce0 2024-09-18T07:14:19.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.589+0000 7f2a49132640 1 -- 172.21.15.70:0/1202730278 >> v1:172.21.15.70:6790/0 conn(0x7f2a440afce0 legacy=0x7f2a4413d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.589+0000 7f2a49132640 1 -- 172.21.15.70:0/1202730278 shutdown_connections 2024-09-18T07:14:19.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.589+0000 7f2a49132640 1 -- 172.21.15.70:0/1202730278 >> 172.21.15.70:0/1202730278 conn(0x7f2a440b1b70 msgr2=0x7f2a440b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:19.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.589+0000 7f2a49132640 1 -- 172.21.15.70:0/1202730278 shutdown_connections 2024-09-18T07:14:19.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.589+0000 7f2a49132640 1 -- 172.21.15.70:0/1202730278 wait complete. 2024-09-18T07:14:19.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.590+0000 7f2a49132640 1 Processor -- start 2024-09-18T07:14:19.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.590+0000 7f2a49132640 1 -- start start 2024-09-18T07:14:19.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a49132640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a440b3700 con 0x7f2a440afce0 2024-09-18T07:14:19.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a49132640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a44135940 con 0x7f2a440ae7f0 2024-09-18T07:14:19.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a49132640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a441330b0 con 0x7f2a44149910 2024-09-18T07:14:19.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2a440afce0 0x7f2a441347c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52200/0 (socket says 172.21.15.70:52200) 2024-09-18T07:14:19.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3bfff640 1 -- 172.21.15.70:0/1429920601 learned_addr learned my addr 172.21.15.70:0/1429920601 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:19.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3081593689 0 0) 0x7f2a441330b0 con 0x7f2a44149910 2024-09-18T07:14:19.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a24003520 con 0x7f2a44149910 2024-09-18T07:14:19.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1403867504 0 0) 0x7f2a440b3700 con 0x7f2a440afce0 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a441330b0 con 0x7f2a440afce0 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 173212558 0 0) 0x7f2a44135940 con 0x7f2a440ae7f0 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a440b3700 con 0x7f2a440ae7f0 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3807590844 0 0) 0x7f2a24003520 con 0x7f2a44149910 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a44135940 con 0x7f2a44149910 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2a3c001b90 con 0x7f2a44149910 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1902622502 0 0) 0x7f2a441330b0 con 0x7f2a440afce0 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a24003520 con 0x7f2a440afce0 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2a2c001840 con 0x7f2a440afce0 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1298150478 0 0) 0x7f2a440b3700 con 0x7f2a440ae7f0 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a441330b0 con 0x7f2a440ae7f0 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2a34001530 con 0x7f2a440ae7f0 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 641965707 0 0) 0x7f2a44135940 con 0x7f2a44149910 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 >> v1:172.21.15.172:6789/0 conn(0x7f2a440ae7f0 legacy=0x7f2a441340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 >> v1:172.21.15.70:6789/0 conn(0x7f2a440afce0 legacy=0x7f2a441347c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4413f8c0 con 0x7f2a44149910 2024-09-18T07:14:19.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a49132640 1 -- 172.21.15.70:0/1429920601 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2a440b3e70 con 0x7f2a44149910 2024-09-18T07:14:19.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.591+0000 7f2a49132640 1 -- 172.21.15.70:0/1429920601 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f2a441e4ae0 con 0x7f2a44149910 2024-09-18T07:14:19.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.592+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2a3c003970 con 0x7f2a44149910 2024-09-18T07:14:19.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.592+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2a3c003b20 con 0x7f2a44149910 2024-09-18T07:14:19.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.592+0000 7f2a49132640 1 -- 172.21.15.70:0/1429920601 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a28005080 con 0x7f2a44149910 2024-09-18T07:14:19.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.593+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f2a3c01c750 con 0x7f2a44149910 2024-09-18T07:14:19.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.594+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f2a3c094f40 con 0x7f2a44149910 2024-09-18T07:14:19.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.598+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f2a3c05d8b0 con 0x7f2a44149910 2024-09-18T07:14:19.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.767+0000 7f2a49132640 1 -- 172.21.15.70:0/1429920601 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a28000b90 con 0x7f2a24079950 2024-09-18T07:14:19.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.768+0000 7f2a3b7fe640 1 -- 172.21.15.70:0/1429920601 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+35848 (unknown 2965378022 0 4039020994) 0x7f2a28000b90 con 0x7f2a24079950 2024-09-18T07:14:19.771 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:14:19.772 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-18T07:14:19.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.770+0000 7f2a49132640 1 -- 172.21.15.70:0/1429920601 >> v1:172.21.15.70:6816/1657990791 conn(0x7f2a24079950 legacy=0x7f2a2407be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.770+0000 7f2a49132640 1 -- 172.21.15.70:0/1429920601 >> v1:172.21.15.70:6790/0 conn(0x7f2a44149910 legacy=0x7f2a441e31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.771+0000 7f2a49132640 1 -- 172.21.15.70:0/1429920601 shutdown_connections 2024-09-18T07:14:19.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.771+0000 7f2a49132640 1 -- 172.21.15.70:0/1429920601 >> 172.21.15.70:0/1429920601 conn(0x7f2a440b1b70 msgr2=0x7f2a4413a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:19.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.771+0000 7f2a49132640 1 -- 172.21.15.70:0/1429920601 shutdown_connections 2024-09-18T07:14:19.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.771+0000 7f2a49132640 1 -- 172.21.15.70:0/1429920601 wait complete. 2024-09-18T07:14:19.784 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-18T07:14:19.446578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":227064,"kb_used_data":2476,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754747656,"statfs":{"total":773094113280,"available":772861599744,"internally_reserved":0,"allocated":2535424,"data_stored":1553494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.206443"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.808451+0000","last_change":"2024-09-18T07:14:15.808642+0000","last_active":"2024-09-18T07:14:15.808451+0000","last_peered":"2024-09-18T07:14:15.808451+0000","last_clean":"2024-09-18T07:14:15.808451+0000","last_became_active":"2024-09-18T07:14:13.251609+0000","last_became_peered":"2024-09-18T07:14:13.251609+0000","last_unstale":"2024-09-18T07:14:15.808451+0000","last_undegraded":"2024-09-18T07:14:15.808451+0000","last_fullsized":"2024-09-18T07:14:15.808451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:01.748551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040903899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.249663+0000","last_change":"2024-09-18T07:14:15.249860+0000","last_active":"2024-09-18T07:14:15.249663+0000","last_peered":"2024-09-18T07:14:15.249663+0000","last_clean":"2024-09-18T07:14:15.249663+0000","last_became_active":"2024-09-18T07:14:13.251585+0000","last_became_peered":"2024-09-18T07:14:13.251585+0000","last_unstale":"2024-09-18T07:14:15.249663+0000","last_undegraded":"2024-09-18T07:14:15.249663+0000","last_fullsized":"2024-09-18T07:14:15.249663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:02:29.914389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039885799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.602830+0000","last_change":"2024-09-18T07:14:15.603034+0000","last_active":"2024-09-18T07:14:15.602830+0000","last_peered":"2024-09-18T07:14:15.602830+0000","last_clean":"2024-09-18T07:14:15.602830+0000","last_became_active":"2024-09-18T07:14:13.957168+0000","last_became_peered":"2024-09-18T07:14:13.957168+0000","last_unstale":"2024-09-18T07:14:15.602830+0000","last_undegraded":"2024-09-18T07:14:15.602830+0000","last_fullsized":"2024-09-18T07:14:15.602830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:57:48.846502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048630800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.249617+0000","last_change":"2024-09-18T07:14:15.249863+0000","last_active":"2024-09-18T07:14:15.249617+0000","last_peered":"2024-09-18T07:14:15.249617+0000","last_clean":"2024-09-18T07:14:15.249617+0000","last_became_active":"2024-09-18T07:14:13.956929+0000","last_became_peered":"2024-09-18T07:14:13.956929+0000","last_unstale":"2024-09-18T07:14:15.249617+0000","last_undegraded":"2024-09-18T07:14:15.249617+0000","last_fullsized":"2024-09-18T07:14:15.249617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:11:55.607274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045339399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.255874+0000","last_change":"2024-09-18T07:14:15.255874+0000","last_active":"2024-09-18T07:14:15.255874+0000","last_peered":"2024-09-18T07:14:15.255874+0000","last_clean":"2024-09-18T07:14:15.255874+0000","last_became_active":"2024-09-18T07:14:13.243579+0000","last_became_peered":"2024-09-18T07:14:13.243579+0000","last_unstale":"2024-09-18T07:14:15.255874+0000","last_undegraded":"2024-09-18T07:14:15.255874+0000","last_fullsized":"2024-09-18T07:14:15.255874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:28.722433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096575000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.398003+0000","last_change":"2024-09-18T07:14:15.398157+0000","last_active":"2024-09-18T07:14:15.398003+0000","last_peered":"2024-09-18T07:14:15.398003+0000","last_clean":"2024-09-18T07:14:15.398003+0000","last_became_active":"2024-09-18T07:14:13.392213+0000","last_became_peered":"2024-09-18T07:14:13.392213+0000","last_unstale":"2024-09-18T07:14:15.398003+0000","last_undegraded":"2024-09-18T07:14:15.398003+0000","last_fullsized":"2024-09-18T07:14:15.398003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:47:04.187501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000312059,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.602800+0000","last_change":"2024-09-18T07:14:15.603018+0000","last_active":"2024-09-18T07:14:15.602800+0000","last_peered":"2024-09-18T07:14:15.602800+0000","last_clean":"2024-09-18T07:14:15.602800+0000","last_became_active":"2024-09-18T07:14:13.251213+0000","last_became_peered":"2024-09-18T07:14:13.251213+0000","last_unstale":"2024-09-18T07:14:15.602800+0000","last_undegraded":"2024-09-18T07:14:15.602800+0000","last_fullsized":"2024-09-18T07:14:15.602800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:43:22.952115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046971900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.255106+0000","last_change":"2024-09-18T07:14:15.255301+0000","last_active":"2024-09-18T07:14:15.255106+0000","last_peered":"2024-09-18T07:14:15.255106+0000","last_clean":"2024-09-18T07:14:15.255106+0000","last_became_active":"2024-09-18T07:14:13.243793+0000","last_became_peered":"2024-09-18T07:14:13.243793+0000","last_unstale":"2024-09-18T07:14:15.255106+0000","last_undegraded":"2024-09-18T07:14:15.255106+0000","last_fullsized":"2024-09-18T07:14:15.255106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:24:14.077910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039863000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-18T07:14:15.602478+0000","last_change":"2024-09-18T07:14:06.881800+0000","last_active":"2024-09-18T07:14:15.602478+0000","last_peered":"2024-09-18T07:14:15.602478+0000","last_clean":"2024-09-18T07:14:15.602478+0000","last_became_active":"2024-09-18T07:14:06.880771+0000","last_became_peered":"2024-09-18T07:14:06.880771+0000","last_unstale":"2024-09-18T07:14:15.602478+0000","last_undegraded":"2024-09-18T07:14:15.602478+0000","last_fullsized":"2024-09-18T07:14:15.602478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:05.472923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:05.472923+0000","last_clean_scrub_stamp":"2024-09-18T07:14:05.472923+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:07.064297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":716800,"data_stored":591467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":57827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":62539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":62539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":184320,"data_stored":62539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":188416,"data_stored":62558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":716800,"data_stored":591467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":188416,"data_stored":62558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T07:14:19.784 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T07:14:19.785 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-18T07:14:19.785 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-18T07:14:19.785 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-18T07:14:19.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.878+0000 7f7a07eb8640 1 Processor -- start 2024-09-18T07:14:19.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.879+0000 7f7a07eb8640 1 -- start start 2024-09-18T07:14:19.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.879+0000 7f7a07eb8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a000b1410 con 0x7f7a000ae7f0 2024-09-18T07:14:19.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.879+0000 7f7a07eb8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a000acaa0 con 0x7f7a000afce0 2024-09-18T07:14:19.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.879+0000 7f7a07eb8640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a0013d8c0 con 0x7f7a00149910 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.879+0000 7f7a06455640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f7a00149910 0x7f7a0014bd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:38046/0 (socket says 172.21.15.70:38046) 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.879+0000 7f7a06455640 1 -- 172.21.15.70:0/865082732 learned_addr learned my addr 172.21.15.70:0/865082732 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.879+0000 7f7a05453640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7a000ae7f0 0x7f7a000abcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52202/0 (socket says 172.21.15.70:52202) 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.880+0000 7f7a04c52640 1 -- 172.21.15.70:0/865082732 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3386363340 0 0) 0x7f7a000b1410 con 0x7f7a000ae7f0 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.880+0000 7f7a04c52640 1 -- 172.21.15.70:0/865082732 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79e8003520 con 0x7f7a000ae7f0 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.880+0000 7f7a04c52640 1 -- 172.21.15.70:0/865082732 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1567533037 0 0) 0x7f7a0013d8c0 con 0x7f7a00149910 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.880+0000 7f7a04c52640 1 -- 172.21.15.70:0/865082732 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a000b1410 con 0x7f7a00149910 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.880+0000 7f7a04c52640 1 -- 172.21.15.70:0/865082732 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 995596238 0 0) 0x7f7a000acaa0 con 0x7f7a000afce0 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.880+0000 7f7a04c52640 1 -- 172.21.15.70:0/865082732 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a0013d8c0 con 0x7f7a000afce0 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.880+0000 7f7a04c52640 1 -- 172.21.15.70:0/865082732 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2557406978 0 0) 0x7f79e8003520 con 0x7f7a000ae7f0 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.880+0000 7f7a04c52640 1 -- 172.21.15.70:0/865082732 >> v1:172.21.15.70:6790/0 conn(0x7f7a00149910 legacy=0x7f7a0014bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.880+0000 7f7a04c52640 1 -- 172.21.15.70:0/865082732 >> v1:172.21.15.172:6789/0 conn(0x7f7a000afce0 legacy=0x7f7a0013d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.880+0000 7f7a04c52640 1 -- 172.21.15.70:0/865082732 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a00074b40 con 0x7f7a000ae7f0 2024-09-18T07:14:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.880+0000 7f7a04c52640 1 -- 172.21.15.70:0/865082732 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f79f4001a10 con 0x7f7a000ae7f0 2024-09-18T07:14:19.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.880+0000 7f7a04c52640 1 -- 172.21.15.70:0/865082732 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f79f4001bb0 con 0x7f7a000ae7f0 2024-09-18T07:14:19.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.880+0000 7f7a04c52640 1 -- 172.21.15.70:0/865082732 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f79f4001fc0 con 0x7f7a000ae7f0 2024-09-18T07:14:19.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.881+0000 7f7a07eb8640 1 -- 172.21.15.70:0/865082732 >> v1:172.21.15.70:6789/0 conn(0x7f7a000ae7f0 legacy=0x7f7a000abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.881+0000 7f7a07eb8640 1 -- 172.21.15.70:0/865082732 shutdown_connections 2024-09-18T07:14:19.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.881+0000 7f7a07eb8640 1 -- 172.21.15.70:0/865082732 >> 172.21.15.70:0/865082732 conn(0x7f7a000b1b70 msgr2=0x7f7a000b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:19.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.881+0000 7f7a07eb8640 1 -- 172.21.15.70:0/865082732 shutdown_connections 2024-09-18T07:14:19.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.881+0000 7f7a07eb8640 1 -- 172.21.15.70:0/865082732 wait complete. 2024-09-18T07:14:19.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.882+0000 7f7a07eb8640 1 Processor -- start 2024-09-18T07:14:19.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.882+0000 7f7a07eb8640 1 -- start start 2024-09-18T07:14:19.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.882+0000 7f7a07eb8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a001420d0 con 0x7f7a00149910 2024-09-18T07:14:19.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.882+0000 7f7a07eb8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a00143850 con 0x7f7a000afce0 2024-09-18T07:14:19.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.882+0000 7f7a07eb8640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a00143b30 con 0x7f7a000ae7f0 2024-09-18T07:14:19.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.882+0000 7f7a05c54640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f7a000ae7f0 0x7f7a001d94b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:38048/0 (socket says 172.21.15.70:38048) 2024-09-18T07:14:19.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.882+0000 7f7a05c54640 1 -- 172.21.15.70:0/1846288489 learned_addr learned my addr 172.21.15.70:0/1846288489 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:19.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.882+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1947681791 0 0) 0x7f7a00143b30 con 0x7f7a000ae7f0 2024-09-18T07:14:19.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.882+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79ec003520 con 0x7f7a000ae7f0 2024-09-18T07:14:19.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.882+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1141696863 0 0) 0x7f7a001420d0 con 0x7f7a00149910 2024-09-18T07:14:19.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.882+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a00143b30 con 0x7f7a00149910 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2544459736 0 0) 0x7f7a00143850 con 0x7f7a000afce0 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a001420d0 con 0x7f7a000afce0 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2524937904 0 0) 0x7f79ec003520 con 0x7f7a000ae7f0 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7a00143850 con 0x7f7a000ae7f0 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f79f0001a10 con 0x7f7a000ae7f0 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2928276616 0 0) 0x7f7a00143b30 con 0x7f7a00149910 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f79ec003520 con 0x7f7a00149910 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f79f80016a0 con 0x7f7a00149910 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2922831939 0 0) 0x7f7a00143850 con 0x7f7a000ae7f0 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 >> v1:172.21.15.172:6789/0 conn(0x7f7a000afce0 legacy=0x7f7a00134e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 >> v1:172.21.15.70:6789/0 conn(0x7f7a00149910 legacy=0x7f7a001e1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a001d9bc0 con 0x7f7a000ae7f0 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f7a07eb8640 1 -- 172.21.15.70:0/1846288489 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7a00135580 con 0x7f7a000ae7f0 2024-09-18T07:14:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f7a07eb8640 1 -- 172.21.15.70:0/1846288489 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f7a00142790 con 0x7f7a000ae7f0 2024-09-18T07:14:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f79f0001d30 con 0x7f7a000ae7f0 2024-09-18T07:14:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.883+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f79f0003aa0 con 0x7f7a000ae7f0 2024-09-18T07:14:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.884+0000 7f7a07eb8640 1 -- 172.21.15.70:0/1846288489 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e0005080 con 0x7f7a000ae7f0 2024-09-18T07:14:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.884+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f79f0003aa0 con 0x7f7a000ae7f0 2024-09-18T07:14:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.885+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f79f0095fd0 con 0x7f7a000ae7f0 2024-09-18T07:14:19.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:19.889+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f79f005e940 con 0x7f7a000ae7f0 2024-09-18T07:14:20.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.097+0000 7f7a07eb8640 1 -- 172.21.15.70:0/1846288489 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f79e0005370 con 0x7f7a000ae7f0 2024-09-18T07:14:20.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.098+0000 7f79fe7fc640 1 -- 172.21.15.70:0/1846288489 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7f79f005e240 con 0x7f7a000ae7f0 2024-09-18T07:14:20.101 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:14:20.101 INFO:teuthology.orchestra.run.smithi070.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-18T07:14:20.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.100+0000 7f7a07eb8640 1 -- 172.21.15.70:0/1846288489 >> v1:172.21.15.70:6816/1657990791 conn(0x7f79ec079670 legacy=0x7f79ec07bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:20.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.100+0000 7f7a07eb8640 1 -- 172.21.15.70:0/1846288489 >> v1:172.21.15.70:6790/0 conn(0x7f7a000ae7f0 legacy=0x7f7a001d94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:20.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.100+0000 7f7a07eb8640 1 -- 172.21.15.70:0/1846288489 shutdown_connections 2024-09-18T07:14:20.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.100+0000 7f7a07eb8640 1 -- 172.21.15.70:0/1846288489 >> 172.21.15.70:0/1846288489 conn(0x7f7a000b1b70 msgr2=0x7f7a00076130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:20.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.100+0000 7f7a07eb8640 1 -- 172.21.15.70:0/1846288489 shutdown_connections 2024-09-18T07:14:20.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.100+0000 7f7a07eb8640 1 -- 172.21.15.70:0/1846288489 wait complete. 2024-09-18T07:14:20.113 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-18T07:14:20.113 INFO:teuthology.run_tasks:Running task exec... 2024-09-18T07:14:20.124 INFO:teuthology.task.exec:Executing custom commands... 2024-09-18T07:14:20.124 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi070.front.sepia.ceph.com 2024-09-18T07:14:20.124 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool create cephsqlite' 2024-09-18T07:14:20.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.218+0000 7f404c12b640 1 Processor -- start 2024-09-18T07:14:20.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.218+0000 7f404c12b640 1 -- start start 2024-09-18T07:14:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f404c12b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4044074f50 con 0x7f40440b0ac0 2024-09-18T07:14:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f404c12b640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4044072290 con 0x7f40440b1400 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f404c12b640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4044072460 con 0x7f404413db40 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f4049ec7640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f40440b0ac0 0x7f4044074840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52228/0 (socket says 172.21.15.70:52228) 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f4049ec7640 1 -- 172.21.15.70:0/1705248658 learned_addr learned my addr 172.21.15.70:0/1705248658 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f404a6c8640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f404413db40 0x7f404414baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:38064/0 (socket says 172.21.15.70:38064) 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f4048ec5640 1 -- 172.21.15.70:0/1705248658 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 564111635 0 0) 0x7f4044074f50 con 0x7f40440b0ac0 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f4048ec5640 1 -- 172.21.15.70:0/1705248658 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4030003520 con 0x7f40440b0ac0 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f4048ec5640 1 -- 172.21.15.70:0/1705248658 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2640269114 0 0) 0x7f4044072460 con 0x7f404413db40 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f4048ec5640 1 -- 172.21.15.70:0/1705248658 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4044074f50 con 0x7f404413db40 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f4048ec5640 1 -- 172.21.15.70:0/1705248658 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2307505509 0 0) 0x7f4030003520 con 0x7f40440b0ac0 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f4048ec5640 1 -- 172.21.15.70:0/1705248658 >> v1:172.21.15.70:6790/0 conn(0x7f404413db40 legacy=0x7f404414baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f4048ec5640 1 -- 172.21.15.70:0/1705248658 >> v1:172.21.15.172:6789/0 conn(0x7f40440b1400 legacy=0x7f404413d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f4048ec5640 1 -- 172.21.15.70:0/1705248658 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40440aea40 con 0x7f40440b0ac0 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f4048ec5640 1 -- 172.21.15.70:0/1705248658 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4040001690 con 0x7f40440b0ac0 2024-09-18T07:14:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f4048ec5640 1 -- 172.21.15.70:0/1705248658 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4040001830 con 0x7f40440b0ac0 2024-09-18T07:14:20.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.219+0000 7f4048ec5640 1 -- 172.21.15.70:0/1705248658 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4040001cc0 con 0x7f40440b0ac0 2024-09-18T07:14:20.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.220+0000 7f404c12b640 1 -- 172.21.15.70:0/1705248658 >> v1:172.21.15.70:6789/0 conn(0x7f40440b0ac0 legacy=0x7f4044074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:20.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.220+0000 7f404c12b640 1 -- 172.21.15.70:0/1705248658 shutdown_connections 2024-09-18T07:14:20.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.220+0000 7f404c12b640 1 -- 172.21.15.70:0/1705248658 >> 172.21.15.70:0/1705248658 conn(0x7f40440b1940 msgr2=0x7f40440b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:20.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.220+0000 7f404c12b640 1 -- 172.21.15.70:0/1705248658 shutdown_connections 2024-09-18T07:14:20.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.220+0000 7f404c12b640 1 -- 172.21.15.70:0/1705248658 wait complete. 2024-09-18T07:14:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.221+0000 7f404c12b640 1 Processor -- start 2024-09-18T07:14:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.221+0000 7f404c12b640 1 -- start start 2024-09-18T07:14:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.221+0000 7f404c12b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40440abc70 con 0x7f404413db40 2024-09-18T07:14:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.221+0000 7f404c12b640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40440aab50 con 0x7f40440b1400 2024-09-18T07:14:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.221+0000 7f404c12b640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40440aad20 con 0x7f40440b0ac0 2024-09-18T07:14:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.221+0000 7f4049ec7640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f40440b0ac0 0x7f4044130eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:38080/0 (socket says 172.21.15.70:38080) 2024-09-18T07:14:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.221+0000 7f4049ec7640 1 -- 172.21.15.70:0/203334080 learned_addr learned my addr 172.21.15.70:0/203334080 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2295677842 0 0) 0x7f40440aad20 con 0x7f40440b0ac0 2024-09-18T07:14:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4028003520 con 0x7f40440b0ac0 2024-09-18T07:14:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 849201863 0 0) 0x7f40440abc70 con 0x7f404413db40 2024-09-18T07:14:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40440aad20 con 0x7f404413db40 2024-09-18T07:14:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 305766700 0 0) 0x7f40440aab50 con 0x7f40440b1400 2024-09-18T07:14:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40440abc70 con 0x7f40440b1400 2024-09-18T07:14:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 684541873 0 0) 0x7f4028003520 con 0x7f40440b0ac0 2024-09-18T07:14:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f40440aab50 con 0x7f40440b0ac0 2024-09-18T07:14:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4040001cc0 con 0x7f40440b0ac0 2024-09-18T07:14:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3779576247 0 0) 0x7f40440aad20 con 0x7f404413db40 2024-09-18T07:14:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4028003520 con 0x7f404413db40 2024-09-18T07:14:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f403c001650 con 0x7f404413db40 2024-09-18T07:14:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3564072408 0 0) 0x7f40440abc70 con 0x7f40440b1400 2024-09-18T07:14:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f40440aad20 con 0x7f40440b1400 2024-09-18T07:14:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4034001870 con 0x7f40440b1400 2024-09-18T07:14:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 41197737 0 0) 0x7f40440aab50 con 0x7f40440b0ac0 2024-09-18T07:14:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 >> v1:172.21.15.172:6789/0 conn(0x7f40440b1400 legacy=0x7f40441d9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 >> v1:172.21.15.70:6789/0 conn(0x7f404413db40 legacy=0x7f40441e1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4044134820 con 0x7f40440b0ac0 2024-09-18T07:14:20.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f404c12b640 1 -- 172.21.15.70:0/203334080 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4044134b10 con 0x7f40440b0ac0 2024-09-18T07:14:20.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f404c12b640 1 -- 172.21.15.70:0/203334080 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f4044070460 con 0x7f40440b0ac0 2024-09-18T07:14:20.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4040001810 con 0x7f40440b0ac0 2024-09-18T07:14:20.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.222+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4040004c30 con 0x7f40440b0ac0 2024-09-18T07:14:20.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.223+0000 7f404c12b640 1 -- 172.21.15.70:0/203334080 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402c005080 con 0x7f40440b0ac0 2024-09-18T07:14:20.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.224+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f4040004c30 con 0x7f40440b0ac0 2024-09-18T07:14:20.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.224+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(21..21 src has 1..21) ==== 4146+0+0 (unknown 1756465126 0 0) 0x7f4040095d90 con 0x7f40440b0ac0 2024-09-18T07:14:20.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.228+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f404005e840 con 0x7f40440b0ac0 2024-09-18T07:14:20.408 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:20.406+0000 7f404c12b640 1 -- 172.21.15.70:0/203334080 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "cephsqlite"} v 0) -- 0x7f402c005370 con 0x7f40440b0ac0 2024-09-18T07:14:21.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.273+0000 7f403a7fc640 1 -- 172.21.15.70:0/203334080 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephsqlite"}]=0 pool 'cephsqlite' created v22) ==== 110+0+0 (unknown 896424662 0 0) 0x7f404005e140 con 0x7f40440b0ac0 2024-09-18T07:14:21.277 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephsqlite' created 2024-09-18T07:14:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.278+0000 7f404c12b640 1 -- 172.21.15.70:0/203334080 >> v1:172.21.15.70:6816/1657990791 conn(0x7f4028079950 legacy=0x7f402807be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.278+0000 7f404c12b640 1 -- 172.21.15.70:0/203334080 >> v1:172.21.15.70:6790/0 conn(0x7f40440b0ac0 legacy=0x7f4044130eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.279+0000 7f404c12b640 1 -- 172.21.15.70:0/203334080 shutdown_connections 2024-09-18T07:14:21.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.279+0000 7f404c12b640 1 -- 172.21.15.70:0/203334080 >> 172.21.15.70:0/203334080 conn(0x7f40440b1940 msgr2=0x7f40440b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:21.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.279+0000 7f404c12b640 1 -- 172.21.15.70:0/203334080 shutdown_connections 2024-09-18T07:14:21.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.279+0000 7f404c12b640 1 -- 172.21.15.70:0/203334080 wait complete. 2024-09-18T07:14:21.294 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph auth get-or-create client.libcephsqlite mon '"'"'profile simple-rados-client-with-blocklist'"'"' osd '"'"'allow rwx pool=cephsqlite'"'"' >> /etc/ceph/ceph.keyring' 2024-09-18T07:14:21.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.398+0000 7f7da2bc6640 1 Processor -- start 2024-09-18T07:14:21.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.398+0000 7f7da2bc6640 1 -- start start 2024-09-18T07:14:21.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.399+0000 7f7da2bc6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9c143b10 con 0x7f7d9c0aabf0 2024-09-18T07:14:21.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.399+0000 7f7da2bc6640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9c140d60 con 0x7f7d9c1436e0 2024-09-18T07:14:21.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.399+0000 7f7da2bc6640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9c0b59c0 con 0x7f7d9c0b6360 2024-09-18T07:14:21.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.399+0000 7f7da0962640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f7d9c0b6360 0x7f7d9c0b3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:46154/0 (socket says 172.21.15.70:46154) 2024-09-18T07:14:21.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.399+0000 7f7da0962640 1 -- 172.21.15.70:0/3123705669 learned_addr learned my addr 172.21.15.70:0/3123705669 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:21.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.399+0000 7f7d9b7fe640 1 -- 172.21.15.70:0/3123705669 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1671422583 0 0) 0x7f7d9c0b59c0 con 0x7f7d9c0b6360 2024-09-18T07:14:21.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.399+0000 7f7d9b7fe640 1 -- 172.21.15.70:0/3123705669 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d84003520 con 0x7f7d9c0b6360 2024-09-18T07:14:21.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.399+0000 7f7d9b7fe640 1 -- 172.21.15.70:0/3123705669 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 895931910 0 0) 0x7f7d9c143b10 con 0x7f7d9c0aabf0 2024-09-18T07:14:21.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.399+0000 7f7d9b7fe640 1 -- 172.21.15.70:0/3123705669 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d9c0b59c0 con 0x7f7d9c0aabf0 2024-09-18T07:14:21.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.399+0000 7f7d9b7fe640 1 -- 172.21.15.70:0/3123705669 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2560352635 0 0) 0x7f7d84003520 con 0x7f7d9c0b6360 2024-09-18T07:14:21.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.399+0000 7f7d9b7fe640 1 -- 172.21.15.70:0/3123705669 >> v1:172.21.15.172:6789/0 conn(0x7f7d9c1436e0 legacy=0x7f7d9c0aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:21.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.399+0000 7f7d9b7fe640 1 -- 172.21.15.70:0/3123705669 >> v1:172.21.15.70:6789/0 conn(0x7f7d9c0aabf0 legacy=0x7f7d9c14eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:21.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.400+0000 7f7d9b7fe640 1 -- 172.21.15.70:0/3123705669 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c0b3720 con 0x7f7d9c0b6360 2024-09-18T07:14:21.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.400+0000 7f7d9b7fe640 1 -- 172.21.15.70:0/3123705669 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7d8c001a10 con 0x7f7d9c0b6360 2024-09-18T07:14:21.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.400+0000 7f7d9b7fe640 1 -- 172.21.15.70:0/3123705669 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7d8c001bb0 con 0x7f7d9c0b6360 2024-09-18T07:14:21.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.400+0000 7f7d9b7fe640 1 -- 172.21.15.70:0/3123705669 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7d8c002040 con 0x7f7d9c0b6360 2024-09-18T07:14:21.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.400+0000 7f7da2bc6640 1 -- 172.21.15.70:0/3123705669 >> v1:172.21.15.70:6790/0 conn(0x7f7d9c0b6360 legacy=0x7f7d9c0b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:21.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.400+0000 7f7da2bc6640 1 -- 172.21.15.70:0/3123705669 shutdown_connections 2024-09-18T07:14:21.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.400+0000 7f7da2bc6640 1 -- 172.21.15.70:0/3123705669 >> 172.21.15.70:0/3123705669 conn(0x7f7d9c074400 msgr2=0x7f7d9c074810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:21.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.400+0000 7f7da2bc6640 1 -- 172.21.15.70:0/3123705669 shutdown_connections 2024-09-18T07:14:21.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.400+0000 7f7da2bc6640 1 -- 172.21.15.70:0/3123705669 wait complete. 2024-09-18T07:14:21.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.401+0000 7f7da2bc6640 1 Processor -- start 2024-09-18T07:14:21.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.401+0000 7f7da2bc6640 1 -- start start 2024-09-18T07:14:21.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.402+0000 7f7da2bc6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9c13b650 con 0x7f7d9c0aabf0 2024-09-18T07:14:21.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.402+0000 7f7da2bc6640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9c1354d0 con 0x7f7d9c0b6360 2024-09-18T07:14:21.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.402+0000 7f7da2bc6640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9c13b2e0 con 0x7f7d9c1436e0 2024-09-18T07:14:21.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.402+0000 7f7da0962640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7d9c0aabf0 0x7f7d9c1332a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44778/0 (socket says 172.21.15.70:44778) 2024-09-18T07:14:21.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.402+0000 7f7da1163640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f7d9c1436e0 0x7f7d9c1d8ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:46164/0 (socket says 172.21.15.70:46164) 2024-09-18T07:14:21.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.402+0000 7f7da0962640 1 -- 172.21.15.70:0/1354617518 learned_addr learned my addr 172.21.15.70:0/1354617518 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:14:21.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.402+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2077953175 0 0) 0x7f7d9c13b2e0 con 0x7f7d9c1436e0 2024-09-18T07:14:21.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.402+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d88003520 con 0x7f7d9c1436e0 2024-09-18T07:14:21.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.402+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3034067070 0 0) 0x7f7d9c13b650 con 0x7f7d9c0aabf0 2024-09-18T07:14:21.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.402+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d9c13b2e0 con 0x7f7d9c0aabf0 2024-09-18T07:14:21.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.402+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1238399309 0 0) 0x7f7d9c1354d0 con 0x7f7d9c0b6360 2024-09-18T07:14:21.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.402+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d9c13b650 con 0x7f7d9c0b6360 2024-09-18T07:14:21.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2301519211 0 0) 0x7f7d88003520 con 0x7f7d9c1436e0 2024-09-18T07:14:21.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7d9c1354d0 con 0x7f7d9c1436e0 2024-09-18T07:14:21.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7d94001310 con 0x7f7d9c1436e0 2024-09-18T07:14:21.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1172694275 0 0) 0x7f7d9c13b2e0 con 0x7f7d9c0aabf0 2024-09-18T07:14:21.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7d88003520 con 0x7f7d9c0aabf0 2024-09-18T07:14:21.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7d8c001f00 con 0x7f7d9c0aabf0 2024-09-18T07:14:21.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3627442327 0 0) 0x7f7d9c1354d0 con 0x7f7d9c1436e0 2024-09-18T07:14:21.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 >> v1:172.21.15.172:6789/0 conn(0x7f7d9c0b6360 legacy=0x7f7d9c1d83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:21.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 >> v1:172.21.15.70:6789/0 conn(0x7f7d9c0aabf0 legacy=0x7f7d9c1332a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:21.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c146180 con 0x7f7d9c1436e0 2024-09-18T07:14:21.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7da2bc6640 1 -- 172.21.15.70:0/1354617518 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9c146470 con 0x7f7d9c1436e0 2024-09-18T07:14:21.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7da2bc6640 1 -- 172.21.15.70:0/1354617518 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f7d9c13bdd0 con 0x7f7d9c1436e0 2024-09-18T07:14:21.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7d940035f0 con 0x7f7d9c1436e0 2024-09-18T07:14:21.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.403+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7d94003a80 con 0x7f7d9c1436e0 2024-09-18T07:14:21.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.404+0000 7f7da2bc6640 1 -- 172.21.15.70:0/1354617518 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d7c005080 con 0x7f7d9c1436e0 2024-09-18T07:14:21.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.404+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f7d94003ca0 con 0x7f7d9c1436e0 2024-09-18T07:14:21.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.405+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(22..22 src has 1..22) ==== 4467+0+0 (unknown 3644098018 0 0) 0x7f7d94096000 con 0x7f7d9c1436e0 2024-09-18T07:14:21.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.409+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f7d9405e970 con 0x7f7d9c1436e0 2024-09-18T07:14:21.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.649+0000 7f7da2bc6640 1 -- 172.21.15.70:0/1354617518 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.libcephsqlite", "caps": ["mon", "profile simple-rados-client-with-blocklist", "osd", "allow rwx pool=cephsqlite"]} v 0) -- 0x7f7d7c005370 con 0x7f7d9c1436e0 2024-09-18T07:14:21.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.653+0000 7f7d997fa640 1 -- 172.21.15.70:0/1354617518 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.libcephsqlite", "caps": ["mon", "profile simple-rados-client-with-blocklist", "osd", "allow rwx pool=cephsqlite"]}]=0 v11) ==== 199+0+71 (unknown 3519214797 0 466221536) 0x7f7d9405e270 con 0x7f7d9c1436e0 2024-09-18T07:14:21.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.655+0000 7f7da2bc6640 1 -- 172.21.15.70:0/1354617518 >> v1:172.21.15.70:6816/1657990791 conn(0x7f7d88079830 legacy=0x7f7d8807bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:21.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.655+0000 7f7da2bc6640 1 -- 172.21.15.70:0/1354617518 >> v1:172.21.15.70:6790/0 conn(0x7f7d9c1436e0 legacy=0x7f7d9c1d8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:14:21.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.655+0000 7f7da2bc6640 1 -- 172.21.15.70:0/1354617518 shutdown_connections 2024-09-18T07:14:21.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.655+0000 7f7da2bc6640 1 -- 172.21.15.70:0/1354617518 >> 172.21.15.70:0/1354617518 conn(0x7f7d9c074400 msgr2=0x7f7d9c0b5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:14:21.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.655+0000 7f7da2bc6640 1 -- 172.21.15.70:0/1354617518 shutdown_connections 2024-09-18T07:14:21.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:14:21.655+0000 7f7da2bc6640 1 -- 172.21.15.70:0/1354617518 wait complete. 2024-09-18T07:14:21.668 INFO:teuthology.run_tasks:Running task exec... 2024-09-18T07:14:21.679 INFO:teuthology.task.exec:Executing custom commands... 2024-09-18T07:14:21.680 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi070.front.sepia.ceph.com 2024-09-18T07:14:21.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr' 2024-09-18T07:14:21.724 INFO:teuthology.orchestra.run.smithi070.stdout:[==========] Running 20 tests from 1 test suite. 2024-09-18T07:14:21.724 INFO:teuthology.orchestra.run.smithi070.stdout:[----------] Global test environment set-up. 2024-09-18T07:14:21.724 INFO:teuthology.orchestra.run.smithi070.stdout:[----------] 20 tests from CephSQLiteTest 2024-09-18T07:14:26.732 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.Create 2024-09-18T07:14:26.743 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/adc738c4-feaf-4842-a642-92bafe13070e.db?vfs=ceph 2024-09-18T07:14:26.743 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:26.748 INFO:teuthology.orchestra.run.smithi070.stdout:CREATE TABLE foo (a INT); 2024-09-18T07:14:26.772 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.Create (41 ms) 2024-09-18T07:14:26.772 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.InsertBulk4096 2024-09-18T07:14:26.782 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/59a35c59-2a3e-4781-b507-347623f7e3dc.db?vfs=ceph 2024-09-18T07:14:26.782 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:26.786 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA page_size = 4096;CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOM() FROM c LIMIT 1000000;PRAGMA page_size; 2024-09-18T07:14:28.401 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.InsertBulk4096 (1629 ms) 2024-09-18T07:14:28.402 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.InsertBulk 2024-09-18T07:14:28.409 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/9dbbe29f-ec02-464d-a2c1-9781a708f742.db?vfs=ceph 2024-09-18T07:14:28.410 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:28.413 INFO:teuthology.orchestra.run.smithi070.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOM() FROM c LIMIT 1000000; 2024-09-18T07:14:28.833 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.InsertBulk (431 ms) 2024-09-18T07:14:28.833 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.UpdateBulk 2024-09-18T07:14:28.840 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/491a17f6-37f7-41a6-955e-38307cc0b4e9.db?vfs=ceph 2024-09-18T07:14:28.841 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:28.844 INFO:teuthology.orchestra.run.smithi070.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;SELECT SUM(a) FROM foo;UPDATE foo SET a = a+a;SELECT SUM(a) FROM foo; 2024-09-18T07:14:29.518 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.UpdateBulk (685 ms) 2024-09-18T07:14:29.518 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.InsertRate 2024-09-18T07:14:29.526 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/848899f6-ebf6-4725-9ae8-ebce60ea8b5b.db?vfs=ceph 2024-09-18T07:14:29.526 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:29.530 INFO:teuthology.orchestra.run.smithi070.stdout:CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-09-18T07:14:31.015 INFO:teuthology.orchestra.run.smithi070.stdout:transactions per second: 68.1683 2024-09-18T07:14:31.016 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.InsertRate (1498 ms) 2024-09-18T07:14:31.016 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.DatabaseShrink 2024-09-18T07:14:31.024 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/c73a860c-3120-4de6-acc6-7fe7540ec3a2.db?vfs=ceph 2024-09-18T07:14:31.024 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;DELETE FROM foo WHERE RANDOM()%4 < 3;VACUUM; 2024-09-18T07:14:32.323 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.DatabaseShrink (1307 ms) 2024-09-18T07:14:32.323 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveRate 2024-09-18T07:14:32.330 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/95f1da89-55a4-40e1-873c-d0134f0a0f8f.db?vfs=ceph 2024-09-18T07:14:32.331 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:32.334 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA locking_mode=EXCLUSIVE;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-09-18T07:14:32.811 INFO:teuthology.orchestra.run.smithi070.stdout:transactions per second: 213.682 2024-09-18T07:14:32.813 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.InsertExclusiveRate (490 ms) 2024-09-18T07:14:32.813 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveWALRate 2024-09-18T07:14:32.821 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/1d08a595-68e0-4a49-83b4-5637473e0d2d.db?vfs=ceph 2024-09-18T07:14:32.821 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:32.825 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA locking_mode=EXCLUSIVE;PRAGMA journal_mode=WAL;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-09-18T07:14:33.120 INFO:teuthology.orchestra.run.smithi070.stdout:transactions per second: 395.269 2024-09-18T07:14:33.133 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.InsertExclusiveWALRate (321 ms) 2024-09-18T07:14:33.133 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.WALTransactionSync 2024-09-18T07:14:33.141 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/1433a28c-e75f-4636-85a5-292b5341a39a.db?vfs=ceph 2024-09-18T07:14:33.141 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:33.144 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA locking_mode=EXCLUSIVE;PRAGMA journal_mode=WAL;CREATE TABLE foo (a INT);INSERT INTO perf (v) VALUES (ceph_perf());BEGIN TRANSACTION;INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount'; 2024-09-18T07:14:33.206 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.WALTransactionSync (73 ms) 2024-09-18T07:14:33.206 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.PersistTransactionSync 2024-09-18T07:14:33.214 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/97510f00-25a7-459d-a014-61a71a95ee97.db?vfs=ceph 2024-09-18T07:14:33.214 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:33.217 INFO:teuthology.orchestra.run.smithi070.stdout:BEGIN TRANSACTION;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount'; 2024-09-18T07:14:33.241 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.PersistTransactionSync (34 ms) 2024-09-18T07:14:33.241 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveLock 2024-09-18T07:14:33.248 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/2314aeee-a58d-4338-a0e1-ac2cd34f6f4b.db?vfs=ceph 2024-09-18T07:14:33.249 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:33.252 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA locking_mode=EXCLUSIVE;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_lock.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_lock.avgcount';SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.lock' AND b.fullkey = '$.libcephsqlite_striper.lock'; 2024-09-18T07:14:33.275 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.InsertExclusiveLock (35 ms) 2024-09-18T07:14:33.275 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.TransactionSizeUpdate 2024-09-18T07:14:33.283 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/fda2d15c-e90c-41d0-90d4-b1ee71703127.db?vfs=ceph 2024-09-18T07:14:33.283 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:33.286 INFO:teuthology.orchestra.run.smithi070.stdout:BEGIN TRANSACTION;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.update_size' AND b.fullkey = '$.libcephsqlite_striper.update_size'; 2024-09-18T07:14:33.303 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.TransactionSizeUpdate (28 ms) 2024-09-18T07:14:33.303 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.AllocatedGrowth 2024-09-18T07:14:33.311 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/e897df03-57b3-46eb-90a5-0a3feb7cabac.db?vfs=ceph 2024-09-18T07:14:33.311 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:33.314 INFO:teuthology.orchestra.run.smithi070.stdout:CREATE TABLE foo (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.update_allocated' AND b.fullkey = '$.libcephsqlite_striper.update_allocated'; 2024-09-18T07:14:40.124 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.AllocatedGrowth (6820 ms) 2024-09-18T07:14:40.124 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.DeleteBulk 2024-09-18T07:14:40.132 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/416be12a-afc4-424a-b73e-8a27e30e3056.db?vfs=ceph 2024-09-18T07:14:40.132 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:40.135 INFO:teuthology.orchestra.run.smithi070.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;DELETE FROM foo WHERE RANDOM()%2 == 0; 2024-09-18T07:14:40.831 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.DeleteBulk (708 ms) 2024-09-18T07:14:40.832 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.DropMassive 2024-09-18T07:14:40.839 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/f0e671ab-0861-46e9-b7da-0a22deba2c91.db?vfs=ceph 2024-09-18T07:14:40.839 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:14:40.843 INFO:teuthology.orchestra.run.smithi070.stdout:CREATE TABLE foo (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;DROP TABLE foo;VACUUM;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.shrink' AND b.fullkey = '$.libcephsqlite_striper.shrink';SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.shrink_bytes' AND b.fullkey = '$.libcephsqlite_striper.shrink_bytes'; 2024-09-18T07:15:08.692 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.DropMassive (27860 ms) 2024-09-18T07:15:08.692 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.InsertMassiveVerify 2024-09-18T07:15:08.701 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/cba38ce1-4a06-4483-98b1-383e495bec11.db?vfs=ceph 2024-09-18T07:15:08.701 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:15:08.706 INFO:teuthology.orchestra.run.smithi070.stdout:CREATE TABLE foo (a BLOB);CREATE TEMPORARY TABLE bar (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO bar (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;SELECT a FROM bar;INSERT INTO foo (a) SELECT a FROM bar;SELECT a FROM foo; 2024-09-18T07:15:21.492 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.InsertMassiveVerify (12800 ms) 2024-09-18T07:15:21.493 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.PerfValid 2024-09-18T07:15:21.501 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/97673491-e549-4450-a8de-6b91a6a01af3.db?vfs=ceph 2024-09-18T07:15:21.502 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:15:21.506 INFO:teuthology.orchestra.run.smithi070.stdout:SELECT json_valid(ceph_perf()); 2024-09-18T07:15:21.507 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.PerfValid (15 ms) 2024-09-18T07:15:21.507 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.StatusValid 2024-09-18T07:15:21.516 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/25cfb0aa-bfd0-48db-a78e-3d6f9d18e720.db?vfs=ceph 2024-09-18T07:15:21.516 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:15:21.522 INFO:teuthology.orchestra.run.smithi070.stdout:SELECT json_valid(ceph_status()); 2024-09-18T07:15:21.523 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.StatusValid (16 ms) 2024-09-18T07:15:21.523 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.CurrentTime 2024-09-18T07:15:21.531 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/44ca1380-7c16-4f3f-b3af-415105c8ccfd.db?vfs=ceph 2024-09-18T07:15:21.532 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:15:21.537 INFO:teuthology.orchestra.run.smithi070.stdout:SELECT strftime('%s', 'now'); 2024-09-18T07:15:21.537 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.CurrentTime (15 ms) 2024-09-18T07:15:21.537 INFO:teuthology.orchestra.run.smithi070.stdout:[ RUN ] CephSQLiteTest.StatusFields 2024-09-18T07:15:21.546 INFO:teuthology.orchestra.run.smithi070.stdout:using database: file:cephsqlite:/25a03f3b-89d1-466b-a7f5-e410a1d833dd.db?vfs=ceph 2024-09-18T07:15:21.546 INFO:teuthology.orchestra.run.smithi070.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-18T07:15:21.551 INFO:teuthology.orchestra.run.smithi070.stdout:SELECT json_extract(ceph_status(), '$.addr');SELECT json_extract(ceph_status(), '$.id'); 2024-09-18T07:15:21.551 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/98414134 2024-09-18T07:15:21.551 INFO:teuthology.orchestra.run.smithi070.stdout:4445 2024-09-18T07:15:21.552 INFO:teuthology.orchestra.run.smithi070.stdout:[ OK ] CephSQLiteTest.StatusFields (14 ms) 2024-09-18T07:15:21.552 INFO:teuthology.orchestra.run.smithi070.stdout:[----------] 20 tests from CephSQLiteTest (54820 ms total) 2024-09-18T07:15:21.552 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:15:21.552 INFO:teuthology.orchestra.run.smithi070.stdout:[----------] Global test environment tear-down 2024-09-18T07:15:21.552 INFO:teuthology.orchestra.run.smithi070.stdout:[==========] 20 tests from 1 test suite ran. (59828 ms total) 2024-09-18T07:15:21.552 INFO:teuthology.orchestra.run.smithi070.stdout:[ PASSED ] 20 tests. 2024-09-18T07:15:21.595 INFO:teuthology.run_tasks:Running task workunit... 2024-09-18T07:15:21.684 INFO:tasks.workunit:Pulling workunits from ref 0325fd3c11a225c0fd9eb38fab88aad1370ad779 2024-09-18T07:15:21.685 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-18T07:15:21.686 DEBUG:teuthology.orchestra.run.smithi070:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-18T07:15:21.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T07:15:21.702 INFO:teuthology.orchestra.run.smithi070.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-18T07:15:21.702 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-18T07:15:21.757 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-18T07:15:21.758 DEBUG:teuthology.orchestra.run.smithi070:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-18T07:15:21.812 INFO:tasks.workunit:timeout=3h 2024-09-18T07:15:21.812 INFO:tasks.workunit:cleanup=True 2024-09-18T07:15:21.813 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 0325fd3c11a225c0fd9eb38fab88aad1370ad779 2024-09-18T07:15:21.868 INFO:tasks.workunit.client.0.smithi070.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-18T07:16:23.563 INFO:tasks.workunit.client.0.smithi070.stderr:Note: switching to '0325fd3c11a225c0fd9eb38fab88aad1370ad779'. 2024-09-18T07:16:23.563 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-09-18T07:16:23.563 INFO:tasks.workunit.client.0.smithi070.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-18T07:16:23.563 INFO:tasks.workunit.client.0.smithi070.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-18T07:16:23.564 INFO:tasks.workunit.client.0.smithi070.stderr:state without impacting any branches by switching back to a branch. 2024-09-18T07:16:23.564 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-09-18T07:16:23.564 INFO:tasks.workunit.client.0.smithi070.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-18T07:16:23.564 INFO:tasks.workunit.client.0.smithi070.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-18T07:16:23.564 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-09-18T07:16:23.564 INFO:tasks.workunit.client.0.smithi070.stderr: git switch -c 2024-09-18T07:16:23.564 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-09-18T07:16:23.564 INFO:tasks.workunit.client.0.smithi070.stderr:Or undo this operation with: 2024-09-18T07:16:23.564 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-09-18T07:16:23.564 INFO:tasks.workunit.client.0.smithi070.stderr: git switch - 2024-09-18T07:16:23.564 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-09-18T07:16:23.565 INFO:tasks.workunit.client.0.smithi070.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-18T07:16:23.565 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-09-18T07:16:23.565 INFO:tasks.workunit.client.0.smithi070.stderr:HEAD is now at 0325fd3c11a Merge pull request #58530 from zdover23/wip-doc-2024-07-11-radosgw-s3-authentication 2024-09-18T07:16:23.565 DEBUG:teuthology.orchestra.run.smithi070:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-18T07:16:23.678 INFO:tasks.workunit.client.0.smithi070.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-18T07:16:23.681 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-18T07:16:23.681 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-18T07:16:24.310 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-18T07:16:24.373 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-18T07:16:24.413 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-18T07:16:24.414 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-18T07:16:24.414 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-18T07:16:24.452 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-18T07:16:24.456 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-18T07:16:24.456 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-18T07:16:24.471 INFO:tasks.workunit:Running workunits matching rados/test_libcephsqlite.sh on client.0... 2024-09-18T07:16:24.472 INFO:tasks.workunit:Running workunit rados/test_libcephsqlite.sh... 2024-09-18T07:16:24.473 DEBUG:teuthology.orchestra.run.smithi070:workunit test rados/test_libcephsqlite.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0325fd3c11a225c0fd9eb38fab88aad1370ad779 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CEPH_ARGS='--id libcephsqlite --no-log-to-stderr' adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh cephsqlite 2024-09-18T07:16:24.530 INFO:tasks.workunit.client.0.smithi070.stderr:+ pool=cephsqlite 2024-09-18T07:16:24.530 INFO:tasks.workunit.client.0.smithi070.stderr:++ basename /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh 2024-09-18T07:16:24.531 INFO:tasks.workunit.client.0.smithi070.stderr:+ ns=test_libcephsqlite.sh 2024-09-18T07:16:24.531 INFO:tasks.workunit.client.0.smithi070.stderr:+ striper rm baz.db 2024-09-18T07:16:24.531 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-09-18T07:16:24.572 INFO:tasks.workunit.client.0.smithi070.stderr:+ sqlite 2024-09-18T07:16:24.572 INFO:tasks.workunit.client.0.smithi070.stderr:+ background= 2024-09-18T07:16:24.572 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' '' = b ']' 2024-09-18T07:16:24.572 INFO:tasks.workunit.client.0.smithi070.stderr:++ cat 2024-09-18T07:16:24.573 INFO:tasks.workunit.client.0.smithi070.stderr:+ a='create table if not exists foo (a INT); 2024-09-18T07:16:24.573 INFO:tasks.workunit.client.0.smithi070.stderr:insert into foo (a) values (RANDOM()); 2024-09-18T07:16:24.573 INFO:tasks.workunit.client.0.smithi070.stderr:drop table foo;' 2024-09-18T07:16:24.573 INFO:tasks.workunit.client.0.smithi070.stderr:+ printf %s 'create table if not exists foo (a INT); 2024-09-18T07:16:24.573 INFO:tasks.workunit.client.0.smithi070.stderr:insert into foo (a) values (RANDOM()); 2024-09-18T07:16:24.573 INFO:tasks.workunit.client.0.smithi070.stderr:drop table foo;' 2024-09-18T07:16:24.574 INFO:tasks.workunit.client.0.smithi070.stderr:create table if not exists foo (a INT); 2024-09-18T07:16:24.574 INFO:tasks.workunit.client.0.smithi070.stderr:insert into foo (a) values (RANDOM()); 2024-09-18T07:16:24.574 INFO:tasks.workunit.client.0.smithi070.stderr:drop table foo;+ '[' '' '!=' b ']' 2024-09-18T07:16:24.574 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 2024-09-18T07:16:24.574 INFO:tasks.workunit.client.0.smithi070.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-18T07:16:24.736 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-09-18T07:16:24.737 INFO:tasks.workunit.client.0.smithi070.stderr:real 0m0.164s 2024-09-18T07:16:24.737 INFO:tasks.workunit.client.0.smithi070.stderr:user 0m0.032s 2024-09-18T07:16:24.737 INFO:tasks.workunit.client.0.smithi070.stderr:sys 0m0.021s 2024-09-18T07:16:24.737 INFO:tasks.workunit.client.0.smithi070.stderr:+ striper stat baz.db 2024-09-18T07:16:24.737 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper stat baz.db 2024-09-18T07:16:24.770 INFO:tasks.workunit.client.0.smithi070.stdout:cephsqlite/baz.db mtime 2024-09-18T07:16:24.000000+0000, size 8192 2024-09-18T07:16:24.772 INFO:tasks.workunit.client.0.smithi070.stderr:+ striper rm baz.db 2024-09-18T07:16:24.772 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-09-18T07:16:24.813 INFO:tasks.workunit.client.0.smithi070.stderr:+ sqlite 2024-09-18T07:16:24.813 INFO:tasks.workunit.client.0.smithi070.stderr:++ repeat 10 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:24.813 INFO:tasks.workunit.client.0.smithi070.stderr:++ n=10 2024-09-18T07:16:24.813 INFO:tasks.workunit.client.0.smithi070.stderr:++ shift 2024-09-18T07:16:24.813 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i = 0 )) 2024-09-18T07:16:24.814 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 10 )) 2024-09-18T07:16:24.814 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:24.814 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:24.814 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 10 )) 2024-09-18T07:16:24.814 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:24.814 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:24.814 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 10 )) 2024-09-18T07:16:24.814 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:24.814 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:24.814 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 10 )) 2024-09-18T07:16:24.814 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:24.814 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:24.814 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 10 )) 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 10 )) 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 10 )) 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 10 )) 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 10 )) 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:24.815 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 10 )) 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 10 )) 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:+ background= 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' '' = b ']' 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:++ cat 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:+ a='CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.816 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:SELECT LENGTH(text) FROM rand; 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:DROP TABLE rand;' 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:+ printf %s 'CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.817 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:SELECT LENGTH(text) FROM rand; 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:DROP TABLE rand;' 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.818 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.819 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:24.819 INFO:tasks.workunit.client.0.smithi070.stderr:SELECT LENGTH(text) FROM rand; 2024-09-18T07:16:24.819 INFO:tasks.workunit.client.0.smithi070.stderr:DROP TABLE rand;+ '[' '' '!=' b ']' 2024-09-18T07:16:24.819 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 2024-09-18T07:16:24.819 INFO:tasks.workunit.client.0.smithi070.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-18T07:16:25.316 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.316 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.316 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.317 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.317 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.317 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.317 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.317 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.317 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.317 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.425 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-09-18T07:16:25.425 INFO:tasks.workunit.client.0.smithi070.stderr:real 0m0.603s 2024-09-18T07:16:25.425 INFO:tasks.workunit.client.0.smithi070.stderr:user 0m0.069s 2024-09-18T07:16:25.425 INFO:tasks.workunit.client.0.smithi070.stderr:sys 0m0.045s 2024-09-18T07:16:25.425 INFO:tasks.workunit.client.0.smithi070.stderr:+ sqlite 2024-09-18T07:16:25.425 INFO:tasks.workunit.client.0.smithi070.stderr:++ repeat 100 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.425 INFO:tasks.workunit.client.0.smithi070.stderr:++ n=100 2024-09-18T07:16:25.425 INFO:tasks.workunit.client.0.smithi070.stderr:++ shift 2024-09-18T07:16:25.425 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i = 0 )) 2024-09-18T07:16:25.425 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.425 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.426 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.426 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.426 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.426 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.426 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.426 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.426 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.426 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.426 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.427 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.427 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.427 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.427 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.427 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.427 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.427 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.427 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.427 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.427 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.427 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.427 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.428 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.428 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.428 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.428 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.428 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.428 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.428 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.428 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.428 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.428 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.428 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.428 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.429 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.430 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.430 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.430 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.430 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.430 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.430 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.430 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.430 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.430 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.430 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.430 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.430 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.430 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.431 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.432 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.432 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.432 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.432 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.432 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.432 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.432 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.432 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.432 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.432 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.432 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.432 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.433 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.433 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.433 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.433 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.433 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.433 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.433 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.433 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.433 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.433 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.433 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.433 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.434 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.434 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.434 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.434 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.434 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.434 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.434 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.434 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.434 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.434 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.434 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.434 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.434 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.435 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.436 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.436 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.436 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.436 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.436 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.436 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.436 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.436 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.436 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.436 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.436 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.436 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.436 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.437 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.437 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.437 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.437 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.437 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.437 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.437 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.437 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.437 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.437 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.437 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.437 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.437 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.438 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.438 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.438 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.438 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.438 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.438 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.438 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.438 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.438 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.438 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.438 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.438 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.438 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.439 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.440 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.441 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.441 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.441 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.441 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.441 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.441 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.441 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.441 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.441 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.441 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.441 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.441 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.441 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.442 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.442 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.442 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.442 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.442 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.442 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.442 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.442 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.442 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.442 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.442 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.442 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.442 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.443 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.444 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.444 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.444 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.444 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.444 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.444 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.444 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.444 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.444 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.444 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.444 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.444 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.444 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.445 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.446 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.446 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.446 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.446 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.446 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.446 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.446 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.446 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.446 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.446 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.446 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.446 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.447 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.447 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.447 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.447 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.447 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.447 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.447 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.447 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.447 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.447 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.447 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.447 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.447 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.448 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.448 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.448 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.448 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.448 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.448 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.448 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.448 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.448 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.448 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.448 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( ++i )) 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:++ (( i < 100 )) 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:+ background= 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' '' = b ']' 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:++ cat 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:+ a='BEGIN TRANSACTION; 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.449 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.450 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.450 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.450 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.450 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.450 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.450 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.450 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.450 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.450 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.450 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.450 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.450 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.450 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.451 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.452 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.453 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.453 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.453 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.453 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.453 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.453 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.453 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.453 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.453 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.453 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.453 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.453 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.453 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.454 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.455 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.455 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.455 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.455 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.455 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.455 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.455 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.455 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.455 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.456 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.456 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.456 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.456 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.456 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.456 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.456 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.456 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.456 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.456 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.456 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.456 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.456 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:COMMIT; 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:SELECT LENGTH(text) FROM rand; 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:DROP TABLE rand;' 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:+ printf %s 'BEGIN TRANSACTION; 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.457 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.458 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.458 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.458 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.458 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.458 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.458 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.458 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.458 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.458 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.458 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.458 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.458 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.458 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.459 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.460 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.460 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.460 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.460 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.460 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.460 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.460 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.460 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.460 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.460 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.460 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.460 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.461 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.461 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.461 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.461 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.461 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.461 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.461 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.461 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.461 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.461 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.461 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.461 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.461 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.462 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.462 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.462 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.462 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.462 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.462 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.462 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.462 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.462 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.462 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.462 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.462 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.462 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.463 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.463 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.463 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.463 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.463 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.463 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.463 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.463 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.463 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.463 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.463 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.463 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.463 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.464 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.464 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.464 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.464 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.464 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.464 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.464 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.464 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.464 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.464 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.464 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.464 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.464 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:COMMIT; 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:SELECT LENGTH(text) FROM rand; 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:DROP TABLE rand;' 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:BEGIN TRANSACTION; 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.465 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.466 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.466 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.466 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.466 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.466 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.466 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.466 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.466 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.466 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.466 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.466 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.466 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.466 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.467 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.467 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.467 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.467 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.467 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.467 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.467 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.467 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.467 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.467 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.467 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.467 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.467 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.468 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.469 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.469 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.469 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.469 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.469 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.469 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.469 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.469 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.469 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.469 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.469 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.469 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.469 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.470 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.470 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.470 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.470 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.470 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.470 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.470 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.470 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.470 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.470 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.471 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.471 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.471 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.471 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.471 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.471 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.471 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.471 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.471 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.471 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.471 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.471 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.471 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.472 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.472 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.472 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.472 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.472 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.472 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.472 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.472 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.472 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.472 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.472 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.472 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.472 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.473 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.473 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.473 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.473 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.473 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.473 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.473 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-18T07:16:25.473 INFO:tasks.workunit.client.0.smithi070.stderr:COMMIT; 2024-09-18T07:16:25.473 INFO:tasks.workunit.client.0.smithi070.stderr:SELECT LENGTH(text) FROM rand; 2024-09-18T07:16:25.473 INFO:tasks.workunit.client.0.smithi070.stderr:DROP TABLE rand;+ '[' '' '!=' b ']' 2024-09-18T07:16:25.473 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 2024-09-18T07:16:25.473 INFO:tasks.workunit.client.0.smithi070.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-18T07:16:25.606 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.606 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.606 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.606 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.606 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.606 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.606 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.606 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.606 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.606 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.606 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.607 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.607 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.607 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.607 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.607 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.607 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.607 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.607 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.607 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.607 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.607 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.607 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.607 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.608 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.608 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.608 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.608 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.608 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.608 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.608 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.608 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.608 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.608 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.608 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.608 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.608 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.609 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.609 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.609 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.609 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.609 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.609 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.609 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.609 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.609 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.609 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.609 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.609 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.609 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.610 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.611 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.612 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.612 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.612 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.612 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.612 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.612 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.612 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.612 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.612 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.612 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.612 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.612 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.612 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.613 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.613 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.613 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.613 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.613 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.613 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.613 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:25.613 INFO:tasks.workunit.client.0.smithi070.stdout:4096 2024-09-18T07:16:26.150 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-09-18T07:16:26.150 INFO:tasks.workunit.client.0.smithi070.stderr:real 0m0.736s 2024-09-18T07:16:26.150 INFO:tasks.workunit.client.0.smithi070.stderr:user 0m0.067s 2024-09-18T07:16:26.150 INFO:tasks.workunit.client.0.smithi070.stderr:sys 0m0.042s 2024-09-18T07:16:26.151 INFO:tasks.workunit.client.0.smithi070.stderr:+ striper rm baz.db 2024-09-18T07:16:26.151 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-09-18T07:16:26.189 INFO:tasks.workunit.client.0.smithi070.stderr:+ date 2024-09-18T07:16:26.189 INFO:tasks.workunit.client.0.smithi070.stdout:Wed Sep 18 07:16:26 AM UTC 2024 2024-09-18T07:16:26.190 INFO:tasks.workunit.client.0.smithi070.stderr:+ sqlite b 2024-09-18T07:16:26.190 INFO:tasks.workunit.client.0.smithi070.stderr:+ background=b 2024-09-18T07:16:26.190 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' b = b ']' 2024-09-18T07:16:26.190 INFO:tasks.workunit.client.0.smithi070.stderr:+ shift 2024-09-18T07:16:26.190 INFO:tasks.workunit.client.0.smithi070.stderr:++ cat 2024-09-18T07:16:26.191 INFO:tasks.workunit.client.0.smithi070.stderr:+ a='CREATE TABLE foo (a BLOB); 2024-09-18T07:16:26.191 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO foo VALUES ("start"); 2024-09-18T07:16:26.191 INFO:tasks.workunit.client.0.smithi070.stderr:WITH RECURSIVE c(x) AS 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr: ( 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr: VALUES(1) 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr: UNION ALL 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr: SELECT x+1 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr: FROM c 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr: ) 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO foo (a) 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr: SELECT RANDOMBLOB(1<<20) 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr: FROM c 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr: LIMIT (1<<20);' 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr:+ printf %s 'CREATE TABLE foo (a BLOB); 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO foo VALUES ("start"); 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr:WITH RECURSIVE c(x) AS 2024-09-18T07:16:26.192 INFO:tasks.workunit.client.0.smithi070.stderr: ( 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr: VALUES(1) 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr: UNION ALL 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr: SELECT x+1 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr: FROM c 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr: ) 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO foo (a) 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr: SELECT RANDOMBLOB(1<<20) 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr: FROM c 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr: LIMIT (1<<20);' 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr:CREATE TABLE foo (a BLOB); 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO foo VALUES ("start"); 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr:WITH RECURSIVE c(x) AS 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr: ( 2024-09-18T07:16:26.193 INFO:tasks.workunit.client.0.smithi070.stderr: VALUES(1) 2024-09-18T07:16:26.194 INFO:tasks.workunit.client.0.smithi070.stderr: UNION ALL 2024-09-18T07:16:26.194 INFO:tasks.workunit.client.0.smithi070.stderr: SELECT x+1 2024-09-18T07:16:26.194 INFO:tasks.workunit.client.0.smithi070.stderr: FROM c 2024-09-18T07:16:26.194 INFO:tasks.workunit.client.0.smithi070.stderr: ) 2024-09-18T07:16:26.194 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO foo (a) 2024-09-18T07:16:26.194 INFO:tasks.workunit.client.0.smithi070.stderr: SELECT RANDOMBLOB(1<<20) 2024-09-18T07:16:26.194 INFO:tasks.workunit.client.0.smithi070.stderr: FROM c 2024-09-18T07:16:26.194 INFO:tasks.workunit.client.0.smithi070.stderr: LIMIT (1<<20);+ '[' b '!=' b ']' 2024-09-18T07:16:26.194 INFO:tasks.workunit.client.0.smithi070.stderr:+ sleep 10 2024-09-18T07:16:26.194 INFO:tasks.workunit.client.0.smithi070.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-18T07:16:36.192 INFO:tasks.workunit.client.0.smithi070.stdout:done 2024-09-18T07:16:36.192 INFO:tasks.workunit.client.0.smithi070.stdout:[1]+ 38762 Running sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" & 2024-09-18T07:16:36.193 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo done 2024-09-18T07:16:36.193 INFO:tasks.workunit.client.0.smithi070.stderr:+ jobs -l 2024-09-18T07:16:36.193 INFO:tasks.workunit.client.0.smithi070.stderr:++ jobs -p 2024-09-18T07:16:36.193 INFO:tasks.workunit.client.0.smithi070.stderr:+ kill -KILL -- 38762 2024-09-18T07:16:36.193 INFO:tasks.workunit.client.0.smithi070.stderr:+ date 2024-09-18T07:16:36.194 INFO:tasks.workunit.client.0.smithi070.stdout:Wed Sep 18 07:16:36 AM UTC 2024 2024-09-18T07:16:36.194 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 2024-09-18T07:16:36.195 INFO:tasks.workunit.client.0.smithi070.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh: line 95: 38762 Killed sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" 2024-09-18T07:16:36.195 INFO:tasks.workunit.client.0.smithi070.stderr:+ date 2024-09-18T07:16:36.196 INFO:tasks.workunit.client.0.smithi070.stdout:Wed Sep 18 07:16:36 AM UTC 2024 2024-09-18T07:16:36.196 INFO:tasks.workunit.client.0.smithi070.stderr:++ sqlite 2024-09-18T07:16:36.196 INFO:tasks.workunit.client.0.smithi070.stderr:++ background= 2024-09-18T07:16:36.196 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' = b ']' 2024-09-18T07:16:36.196 INFO:tasks.workunit.client.0.smithi070.stderr:+++ cat 2024-09-18T07:16:36.197 INFO:tasks.workunit.client.0.smithi070.stderr:++ a='SELECT COUNT(*) FROM foo;' 2024-09-18T07:16:36.197 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %s 'SELECT COUNT(*) FROM foo;' 2024-09-18T07:16:36.197 INFO:tasks.workunit.client.0.smithi070.stderr:SELECT COUNT(*) FROM foo;++ '[' '' '!=' b ']' 2024-09-18T07:16:36.198 INFO:tasks.workunit.client.0.smithi070.stderr:++ wait 2024-09-18T07:16:36.198 INFO:tasks.workunit.client.0.smithi070.stderr:++ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-18T07:17:03.207 INFO:tasks.workunit.client.0.smithi070.stderr:+ n=1 2024-09-18T07:17:03.207 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-09-18T07:17:03.207 INFO:tasks.workunit.client.0.smithi070.stderr:+ striper rm baz.db 2024-09-18T07:17:03.208 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-09-18T07:17:03.250 INFO:tasks.workunit.client.0.smithi070.stderr:+ date 2024-09-18T07:17:03.250 INFO:tasks.workunit.client.0.smithi070.stdout:Wed Sep 18 07:17:03 AM UTC 2024 2024-09-18T07:17:03.251 INFO:tasks.workunit.client.0.smithi070.stderr:+ sqlite b 2024-09-18T07:17:03.251 INFO:tasks.workunit.client.0.smithi070.stderr:+ background=b 2024-09-18T07:17:03.251 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' b = b ']' 2024-09-18T07:17:03.251 INFO:tasks.workunit.client.0.smithi070.stderr:+ shift 2024-09-18T07:17:03.251 INFO:tasks.workunit.client.0.smithi070.stderr:++ cat 2024-09-18T07:17:03.252 INFO:tasks.workunit.client.0.smithi070.stderr:+ a='CREATE TABLE foo (a BLOB); 2024-09-18T07:17:03.252 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO foo VALUES ("start"); 2024-09-18T07:17:03.252 INFO:tasks.workunit.client.0.smithi070.stderr:WITH RECURSIVE c(x) AS 2024-09-18T07:17:03.253 INFO:tasks.workunit.client.0.smithi070.stderr: ( 2024-09-18T07:17:03.253 INFO:tasks.workunit.client.0.smithi070.stderr: VALUES(1) 2024-09-18T07:17:03.253 INFO:tasks.workunit.client.0.smithi070.stderr: UNION ALL 2024-09-18T07:17:03.253 INFO:tasks.workunit.client.0.smithi070.stderr: SELECT x+1 2024-09-18T07:17:03.253 INFO:tasks.workunit.client.0.smithi070.stderr: FROM c 2024-09-18T07:17:03.253 INFO:tasks.workunit.client.0.smithi070.stderr: ) 2024-09-18T07:17:03.253 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO foo (a) 2024-09-18T07:17:03.253 INFO:tasks.workunit.client.0.smithi070.stderr: SELECT RANDOMBLOB(1<<20) 2024-09-18T07:17:03.253 INFO:tasks.workunit.client.0.smithi070.stderr: FROM c 2024-09-18T07:17:03.253 INFO:tasks.workunit.client.0.smithi070.stderr: LIMIT (1<<20);' 2024-09-18T07:17:03.253 INFO:tasks.workunit.client.0.smithi070.stderr:+ printf %s 'CREATE TABLE foo (a BLOB); 2024-09-18T07:17:03.253 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO foo VALUES ("start"); 2024-09-18T07:17:03.254 INFO:tasks.workunit.client.0.smithi070.stderr:WITH RECURSIVE c(x) AS 2024-09-18T07:17:03.254 INFO:tasks.workunit.client.0.smithi070.stderr: ( 2024-09-18T07:17:03.254 INFO:tasks.workunit.client.0.smithi070.stderr: VALUES(1) 2024-09-18T07:17:03.254 INFO:tasks.workunit.client.0.smithi070.stderr: UNION ALL 2024-09-18T07:17:03.254 INFO:tasks.workunit.client.0.smithi070.stderr: SELECT x+1 2024-09-18T07:17:03.254 INFO:tasks.workunit.client.0.smithi070.stderr: FROM c 2024-09-18T07:17:03.254 INFO:tasks.workunit.client.0.smithi070.stderr: ) 2024-09-18T07:17:03.254 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO foo (a) 2024-09-18T07:17:03.254 INFO:tasks.workunit.client.0.smithi070.stderr: SELECT RANDOMBLOB(1<<20) 2024-09-18T07:17:03.255 INFO:tasks.workunit.client.0.smithi070.stderr: FROM c 2024-09-18T07:17:03.255 INFO:tasks.workunit.client.0.smithi070.stderr: LIMIT (1<<20);' 2024-09-18T07:17:03.255 INFO:tasks.workunit.client.0.smithi070.stderr:CREATE TABLE foo (a BLOB); 2024-09-18T07:17:03.255 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO foo VALUES ("start"); 2024-09-18T07:17:03.255 INFO:tasks.workunit.client.0.smithi070.stderr:WITH RECURSIVE c(x) AS 2024-09-18T07:17:03.255 INFO:tasks.workunit.client.0.smithi070.stderr: ( 2024-09-18T07:17:03.255 INFO:tasks.workunit.client.0.smithi070.stderr: VALUES(1) 2024-09-18T07:17:03.255 INFO:tasks.workunit.client.0.smithi070.stderr: UNION ALL 2024-09-18T07:17:03.255 INFO:tasks.workunit.client.0.smithi070.stderr: SELECT x+1 2024-09-18T07:17:03.255 INFO:tasks.workunit.client.0.smithi070.stderr: FROM c 2024-09-18T07:17:03.255 INFO:tasks.workunit.client.0.smithi070.stderr: ) 2024-09-18T07:17:03.256 INFO:tasks.workunit.client.0.smithi070.stderr:INSERT INTO foo (a) 2024-09-18T07:17:03.256 INFO:tasks.workunit.client.0.smithi070.stderr: SELECT RANDOMBLOB(1<<20) 2024-09-18T07:17:03.256 INFO:tasks.workunit.client.0.smithi070.stderr: FROM c 2024-09-18T07:17:03.256 INFO:tasks.workunit.client.0.smithi070.stderr: LIMIT (1<<20);+ '[' b '!=' b ']' 2024-09-18T07:17:03.256 INFO:tasks.workunit.client.0.smithi070.stderr:+ sleep 20 2024-09-18T07:17:03.256 INFO:tasks.workunit.client.0.smithi070.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-18T07:17:23.254 INFO:tasks.workunit.client.0.smithi070.stderr:+ jobs -l 2024-09-18T07:17:23.254 INFO:tasks.workunit.client.0.smithi070.stdout:[1]+ 38830 Running sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" & 2024-09-18T07:17:23.255 INFO:tasks.workunit.client.0.smithi070.stderr:++ jobs -p 2024-09-18T07:17:23.255 INFO:tasks.workunit.client.0.smithi070.stderr:+ kill -STOP -- 38830 2024-09-18T07:17:23.255 INFO:tasks.workunit.client.0.smithi070.stderr:+ sleep 45 2024-09-18T07:18:08.256 INFO:tasks.workunit.client.0.smithi070.stderr:+ date 2024-09-18T07:18:08.257 INFO:tasks.workunit.client.0.smithi070.stdout:Wed Sep 18 07:18:08 AM UTC 2024 2024-09-18T07:18:08.257 INFO:tasks.workunit.client.0.smithi070.stderr:++ jobs -p 2024-09-18T07:18:08.257 INFO:tasks.workunit.client.0.smithi070.stderr:+ kill -CONT -- 38830 2024-09-18T07:18:08.257 INFO:tasks.workunit.client.0.smithi070.stderr:+ sleep 10 2024-09-18T07:18:08.277 INFO:tasks.workunit.client.0.smithi070.stderr:Error: near line 3: disk I/O error 2024-09-18T07:18:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:+ date 2024-09-18T07:18:18.259 INFO:tasks.workunit.client.0.smithi070.stdout:Wed Sep 18 07:18:18 AM UTC 2024 2024-09-18T07:18:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 2024-09-18T07:18:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:+ date 2024-09-18T07:18:18.260 INFO:tasks.workunit.client.0.smithi070.stdout:Wed Sep 18 07:18:18 AM UTC 2024 2024-09-18T07:18:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:++ sqlite 2024-09-18T07:18:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:++ background= 2024-09-18T07:18:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' = b ']' 2024-09-18T07:18:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:+++ cat 2024-09-18T07:18:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ a='SELECT COUNT(*) FROM foo;' 2024-09-18T07:18:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %s 'SELECT COUNT(*) FROM foo;' 2024-09-18T07:18:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:SELECT COUNT(*) FROM foo;++ '[' '' '!=' b ']' 2024-09-18T07:18:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ wait 2024-09-18T07:18:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-18T07:18:19.312 INFO:tasks.workunit.client.0.smithi070.stderr:+ n=1 2024-09-18T07:18:19.312 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-09-18T07:18:19.313 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-18T07:18:19.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-18T07:18:19.379 INFO:tasks.workunit:Stopping ['rados/test_libcephsqlite.sh cephsqlite'] on client.0... 2024-09-18T07:18:19.379 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-18T07:18:19.691 DEBUG:teuthology.parallel:result is None 2024-09-18T07:18:19.692 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-18T07:18:19.716 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-18T07:18:19.716 DEBUG:teuthology.orchestra.run.smithi070:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-18T07:18:19.777 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-18T07:18:19.778 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-18T07:18:19.788 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T07:18:19.788 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T07:18:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.912+0000 7fbd02de2640 1 Processor -- start 2024-09-18T07:18:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.912+0000 7fbd02de2640 1 -- start start 2024-09-18T07:18:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.913+0000 7fbd02de2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcfc074fe0 con 0x7fbcfc142db0 2024-09-18T07:18:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.913+0000 7fbd02de2640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcfc073f60 con 0x7fbcfc074320 2024-09-18T07:18:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.913+0000 7fbd02de2640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcfc144b70 con 0x7fbcfc147350 2024-09-18T07:18:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.913+0000 7fbd0137f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbcfc142db0 0x7fbcfc143240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34128/0 (socket says 172.21.15.70:34128) 2024-09-18T07:18:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.913+0000 7fbd00b7e640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fbcfc147350 0x7fbcfc071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:36404/0 (socket says 172.21.15.70:36404) 2024-09-18T07:18:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.913+0000 7fbd0137f640 1 -- 172.21.15.70:0/347979512 learned_addr learned my addr 172.21.15.70:0/347979512 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:19.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.913+0000 7fbcfb7fe640 1 -- 172.21.15.70:0/347979512 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2781550033 0 0) 0x7fbcfc074fe0 con 0x7fbcfc142db0 2024-09-18T07:18:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbcfb7fe640 1 -- 172.21.15.70:0/347979512 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbce8003520 con 0x7fbcfc142db0 2024-09-18T07:18:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbcfb7fe640 1 -- 172.21.15.70:0/347979512 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2984985486 0 0) 0x7fbcfc144b70 con 0x7fbcfc147350 2024-09-18T07:18:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbcfb7fe640 1 -- 172.21.15.70:0/347979512 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcfc074fe0 con 0x7fbcfc147350 2024-09-18T07:18:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbcfb7fe640 1 -- 172.21.15.70:0/347979512 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 815169183 0 0) 0x7fbce8003520 con 0x7fbcfc142db0 2024-09-18T07:18:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbcfb7fe640 1 -- 172.21.15.70:0/347979512 >> v1:172.21.15.70:6790/0 conn(0x7fbcfc147350 legacy=0x7fbcfc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbcfb7fe640 1 -- 172.21.15.70:0/347979512 >> v1:172.21.15.172:6789/0 conn(0x7fbcfc074320 legacy=0x7fbcfc147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbcfb7fe640 1 -- 172.21.15.70:0/347979512 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc147e40 con 0x7fbcfc142db0 2024-09-18T07:18:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbcfb7fe640 1 -- 172.21.15.70:0/347979512 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fbcf4001690 con 0x7fbcfc142db0 2024-09-18T07:18:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbcfb7fe640 1 -- 172.21.15.70:0/347979512 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbcf40018b0 con 0x7fbcfc142db0 2024-09-18T07:18:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbcfb7fe640 1 -- 172.21.15.70:0/347979512 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fbcf4001d40 con 0x7fbcfc142db0 2024-09-18T07:18:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbd02de2640 1 -- 172.21.15.70:0/347979512 >> v1:172.21.15.70:6789/0 conn(0x7fbcfc142db0 legacy=0x7fbcfc143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:19.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbd02de2640 1 -- 172.21.15.70:0/347979512 shutdown_connections 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbd02de2640 1 -- 172.21.15.70:0/347979512 >> 172.21.15.70:0/347979512 conn(0x7fbcfc0b64d0 msgr2=0x7fbcfc0b68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbd02de2640 1 -- 172.21.15.70:0/347979512 shutdown_connections 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.914+0000 7fbd02de2640 1 -- 172.21.15.70:0/347979512 wait complete. 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.915+0000 7fbd02de2640 1 Processor -- start 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.915+0000 7fbd02de2640 1 -- start start 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.916+0000 7fbd02de2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcfc1d8b40 con 0x7fbcfc142db0 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.916+0000 7fbd02de2640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcfc14f6d0 con 0x7fbcfc147350 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.916+0000 7fbd02de2640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcfc13b5f0 con 0x7fbcfc074320 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.916+0000 7fbd00b7e640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fbcfc074320 0x7fbcfc13e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:36410/0 (socket says 172.21.15.70:36410) 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.916+0000 7fbd00b7e640 1 -- 172.21.15.70:0/3133524921 learned_addr learned my addr 172.21.15.70:0/3133524921 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.916+0000 7fbcfbfff640 1 --1- 172.21.15.70:0/3133524921 >> v1:172.21.15.70:6789/0 conn(0x7fbcfc142db0 0x7fbcfc1d9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34134/0 (socket says 172.21.15.70:34134) 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.916+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2474785392 0 0) 0x7fbcfc1d8b40 con 0x7fbcfc142db0 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.916+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcdc003520 con 0x7fbcfc142db0 2024-09-18T07:18:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.916+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3859990215 0 0) 0x7fbcfc13b5f0 con 0x7fbcfc074320 2024-09-18T07:18:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.916+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcfc1d8b40 con 0x7fbcfc074320 2024-09-18T07:18:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3902063774 0 0) 0x7fbcdc003520 con 0x7fbcfc142db0 2024-09-18T07:18:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbcfc13b5f0 con 0x7fbcfc142db0 2024-09-18T07:18:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fbce40018f0 con 0x7fbcfc142db0 2024-09-18T07:18:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 524685457 0 0) 0x7fbcfc1d8b40 con 0x7fbcfc074320 2024-09-18T07:18:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbcdc003520 con 0x7fbcfc074320 2024-09-18T07:18:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fbcec001bf0 con 0x7fbcfc074320 2024-09-18T07:18:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3285879284 0 0) 0x7fbcfc14f6d0 con 0x7fbcfc147350 2024-09-18T07:18:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcfc1d8b40 con 0x7fbcfc147350 2024-09-18T07:18:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2206118275 0 0) 0x7fbcfc13b5f0 con 0x7fbcfc142db0 2024-09-18T07:18:19.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 >> v1:172.21.15.70:6790/0 conn(0x7fbcfc074320 legacy=0x7fbcfc13e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 >> v1:172.21.15.172:6789/0 conn(0x7fbcfc147350 legacy=0x7fbcfc138a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc1d9770 con 0x7fbcfc142db0 2024-09-18T07:18:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbd02de2640 1 -- 172.21.15.70:0/3133524921 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbcfc1d9a60 con 0x7fbcfc142db0 2024-09-18T07:18:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbd02de2640 1 -- 172.21.15.70:0/3133524921 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fbcfc139360 con 0x7fbcfc142db0 2024-09-18T07:18:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbce4003610 con 0x7fbcfc142db0 2024-09-18T07:18:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.917+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fbce4003a20 con 0x7fbcfc142db0 2024-09-18T07:18:19.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.918+0000 7fbd02de2640 1 -- 172.21.15.70:0/3133524921 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce0005080 con 0x7fbcfc142db0 2024-09-18T07:18:19.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.919+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fbce4003bc0 con 0x7fbcfc142db0 2024-09-18T07:18:19.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.919+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7fbce4096190 con 0x7fbcfc142db0 2024-09-18T07:18:19.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:19.923+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7fbce405e990 con 0x7fbcfc142db0 2024-09-18T07:18:20.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.096+0000 7fbd02de2640 1 -- 172.21.15.70:0/3133524921 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbce0000b90 con 0x7fbcdc0798c0 2024-09-18T07:18:20.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.099+0000 7fbcf97fa640 1 -- 172.21.15.70:0/3133524921 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117442 (unknown 2965378022 0 272469850) 0x7fbce0000b90 con 0x7fbcdc0798c0 2024-09-18T07:18:20.103 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:18:20.103 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-18T07:18:20.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.101+0000 7fbd02de2640 1 -- 172.21.15.70:0/3133524921 >> v1:172.21.15.70:6816/1657990791 conn(0x7fbcdc0798c0 legacy=0x7fbcdc07bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.101+0000 7fbd02de2640 1 -- 172.21.15.70:0/3133524921 >> v1:172.21.15.70:6789/0 conn(0x7fbcfc142db0 legacy=0x7fbcfc1d9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.102+0000 7fbd02de2640 1 -- 172.21.15.70:0/3133524921 shutdown_connections 2024-09-18T07:18:20.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.102+0000 7fbd02de2640 1 -- 172.21.15.70:0/3133524921 >> 172.21.15.70:0/3133524921 conn(0x7fbcfc0b64d0 msgr2=0x7fbcfc146650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:20.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.102+0000 7fbd02de2640 1 -- 172.21.15.70:0/3133524921 shutdown_connections 2024-09-18T07:18:20.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.102+0000 7fbd02de2640 1 -- 172.21.15.70:0/3133524921 wait complete. 2024-09-18T07:18:20.115 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-18T07:18:19.496357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3582297403,"num_objects":886,"num_object_clones":0,"num_object_copies":1772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":886,"num_whiteouts":0,"num_read":2575,"num_read_kb":2575,"num_write":165428,"num_write_kb":5660663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":168239,"ondisk_log_size":168239,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8574080,"kb_used_data":6707712,"kb_used_omap":12,"kb_used_meta":1866227,"kb_avail":746400640,"statfs":{"total":773094113280,"available":764314255360,"internally_reserved":0,"allocated":6868697088,"data_stored":6863196390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1911017008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":69632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002343"},"pg_stats":[{"pgid":"3.1f","version":"24'3617","reported_seq":3635,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.171919+0000","last_change":"2024-09-18T07:14:22.286661+0000","last_active":"2024-09-18T07:17:03.171919+0000","last_peered":"2024-09-18T07:17:03.171919+0000","last_clean":"2024-09-18T07:17:03.171919+0000","last_became_active":"2024-09-18T07:14:22.286401+0000","last_became_peered":"2024-09-18T07:14:22.286401+0000","last_unstale":"2024-09-18T07:17:03.171919+0000","last_undegraded":"2024-09-18T07:17:03.171919+0000","last_fullsized":"2024-09-18T07:17:03.171919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3617,"log_dups_size":0,"ondisk_log_size":3617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:26:38.664961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95160672,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":3623,"num_write_kb":151525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"25'6414","reported_seq":6531,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:09.318453+0000","last_change":"2024-09-18T07:14:22.284740+0000","last_active":"2024-09-18T07:17:09.318453+0000","last_peered":"2024-09-18T07:17:09.318453+0000","last_clean":"2024-09-18T07:17:09.318453+0000","last_became_active":"2024-09-18T07:14:22.284487+0000","last_became_peered":"2024-09-18T07:14:22.284487+0000","last_unstale":"2024-09-18T07:17:09.318453+0000","last_undegraded":"2024-09-18T07:17:09.318453+0000","last_fullsized":"2024-09-18T07:17:09.318453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":6414,"log_dups_size":0,"ondisk_log_size":6414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:23:39.072202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377808,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":6397,"num_write_kb":190429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"25'7587","reported_seq":7636,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:14.357682+0000","last_change":"2024-09-18T07:14:22.288902+0000","last_active":"2024-09-18T07:17:14.357682+0000","last_peered":"2024-09-18T07:17:14.357682+0000","last_clean":"2024-09-18T07:17:14.357682+0000","last_became_active":"2024-09-18T07:14:22.288441+0000","last_became_peered":"2024-09-18T07:14:22.288441+0000","last_unstale":"2024-09-18T07:17:14.357682+0000","last_undegraded":"2024-09-18T07:17:14.357682+0000","last_fullsized":"2024-09-18T07:17:14.357682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7587,"log_dups_size":0,"ondisk_log_size":7587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:04:30.016419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167903232,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":127,"num_read_kb":103,"num_write":7620,"num_write_kb":243849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"24'3255","reported_seq":3290,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138103+0000","last_change":"2024-09-18T07:14:22.288842+0000","last_active":"2024-09-18T07:17:03.138103+0000","last_peered":"2024-09-18T07:17:03.138103+0000","last_clean":"2024-09-18T07:17:03.138103+0000","last_became_active":"2024-09-18T07:14:22.287853+0000","last_became_peered":"2024-09-18T07:14:22.287853+0000","last_unstale":"2024-09-18T07:17:03.138103+0000","last_undegraded":"2024-09-18T07:17:03.138103+0000","last_fullsized":"2024-09-18T07:17:03.138103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3255,"log_dups_size":0,"ondisk_log_size":3255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:14.436120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88211456,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":79,"num_read_kb":61,"num_write":3296,"num_write_kb":156739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"25'3829","reported_seq":3848,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:18:08.280825+0000","last_change":"2024-09-18T07:14:22.286284+0000","last_active":"2024-09-18T07:18:08.280825+0000","last_peered":"2024-09-18T07:18:08.280825+0000","last_clean":"2024-09-18T07:18:08.280825+0000","last_became_active":"2024-09-18T07:14:22.286018+0000","last_became_peered":"2024-09-18T07:14:22.286018+0000","last_unstale":"2024-09-18T07:18:08.280825+0000","last_undegraded":"2024-09-18T07:18:08.280825+0000","last_fullsized":"2024-09-18T07:18:08.280825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3829,"log_dups_size":0,"ondisk_log_size":3829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:03:53.111506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91650400,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":3788,"num_write_kb":184441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"25'5565","reported_seq":5585,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:07.526513+0000","last_change":"2024-09-18T07:14:22.280275+0000","last_active":"2024-09-18T07:17:07.526513+0000","last_peered":"2024-09-18T07:17:07.526513+0000","last_clean":"2024-09-18T07:17:07.526513+0000","last_became_active":"2024-09-18T07:14:22.279826+0000","last_became_peered":"2024-09-18T07:14:22.279826+0000","last_unstale":"2024-09-18T07:17:07.526513+0000","last_undegraded":"2024-09-18T07:17:07.526513+0000","last_fullsized":"2024-09-18T07:17:07.526513+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5565,"log_dups_size":0,"ondisk_log_size":5565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:19:40.864770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":15,"num_read_kb":75,"num_write":5550,"num_write_kb":155179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"25'4743","reported_seq":4775,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:18.223911+0000","last_change":"2024-09-18T07:14:22.280164+0000","last_active":"2024-09-18T07:17:18.223911+0000","last_peered":"2024-09-18T07:17:18.223911+0000","last_clean":"2024-09-18T07:17:18.223911+0000","last_became_active":"2024-09-18T07:14:22.279885+0000","last_became_peered":"2024-09-18T07:14:22.279885+0000","last_unstale":"2024-09-18T07:17:18.223911+0000","last_undegraded":"2024-09-18T07:17:18.223911+0000","last_fullsized":"2024-09-18T07:17:18.223911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:08:22.395662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":4713,"num_write_kb":147181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"25'8007","reported_seq":8024,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:18.366347+0000","last_change":"2024-09-18T07:14:22.283501+0000","last_active":"2024-09-18T07:17:18.366347+0000","last_peered":"2024-09-18T07:17:18.366347+0000","last_clean":"2024-09-18T07:17:18.366347+0000","last_became_active":"2024-09-18T07:14:22.283231+0000","last_became_peered":"2024-09-18T07:14:22.283231+0000","last_unstale":"2024-09-18T07:17:18.366347+0000","last_undegraded":"2024-09-18T07:17:18.366347+0000","last_fullsized":"2024-09-18T07:17:18.366347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":8007,"log_dups_size":0,"ondisk_log_size":8007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:49:47.239612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125895176,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":8013,"num_write_kb":214525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"25'7235","reported_seq":7255,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:21.527844+0000","last_change":"2024-09-18T07:14:22.286153+0000","last_active":"2024-09-18T07:17:21.527844+0000","last_peered":"2024-09-18T07:17:21.527844+0000","last_clean":"2024-09-18T07:17:21.527844+0000","last_became_active":"2024-09-18T07:14:22.285884+0000","last_became_peered":"2024-09-18T07:14:22.285884+0000","last_unstale":"2024-09-18T07:17:21.527844+0000","last_undegraded":"2024-09-18T07:17:21.527844+0000","last_fullsized":"2024-09-18T07:17:21.527844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7235,"log_dups_size":0,"ondisk_log_size":7235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:36:41.719570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135659520,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":7238,"num_write_kb":200827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"25'3408","reported_seq":3432,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.319245+0000","last_change":"2024-09-18T07:14:22.289679+0000","last_active":"2024-09-18T07:17:03.319245+0000","last_peered":"2024-09-18T07:17:03.319245+0000","last_clean":"2024-09-18T07:17:03.319245+0000","last_became_active":"2024-09-18T07:14:22.289372+0000","last_became_peered":"2024-09-18T07:14:22.289372+0000","last_unstale":"2024-09-18T07:17:03.319245+0000","last_undegraded":"2024-09-18T07:17:03.319245+0000","last_fullsized":"2024-09-18T07:17:03.319245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3408,"log_dups_size":0,"ondisk_log_size":3408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:59:31.222383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112003944,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":22,"num_read_kb":143,"num_write":3412,"num_write_kb":151531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"25'8025","reported_seq":8044,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:17.126686+0000","last_change":"2024-09-18T07:14:22.288942+0000","last_active":"2024-09-18T07:17:17.126686+0000","last_peered":"2024-09-18T07:17:17.126686+0000","last_clean":"2024-09-18T07:17:17.126686+0000","last_became_active":"2024-09-18T07:14:22.288634+0000","last_became_peered":"2024-09-18T07:14:22.288634+0000","last_unstale":"2024-09-18T07:17:17.126686+0000","last_undegraded":"2024-09-18T07:17:17.126686+0000","last_fullsized":"2024-09-18T07:17:17.126686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":8025,"log_dups_size":0,"ondisk_log_size":8025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:34:15.636211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8027,"num_write_kb":201731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"25'4872","reported_seq":4889,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:10.976186+0000","last_change":"2024-09-18T07:14:22.291854+0000","last_active":"2024-09-18T07:17:10.976186+0000","last_peered":"2024-09-18T07:17:10.976186+0000","last_clean":"2024-09-18T07:17:10.976186+0000","last_became_active":"2024-09-18T07:14:22.291551+0000","last_became_peered":"2024-09-18T07:14:22.291551+0000","last_unstale":"2024-09-18T07:17:10.976186+0000","last_undegraded":"2024-09-18T07:17:10.976186+0000","last_fullsized":"2024-09-18T07:17:10.976186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4872,"log_dups_size":0,"ondisk_log_size":4872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:54:13.729656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303680,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4826,"num_write_kb":160133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.171591+0000","last_change":"2024-09-18T07:14:15.808642+0000","last_active":"2024-09-18T07:17:03.171591+0000","last_peered":"2024-09-18T07:17:03.171591+0000","last_clean":"2024-09-18T07:17:03.171591+0000","last_became_active":"2024-09-18T07:14:13.251609+0000","last_became_peered":"2024-09-18T07:14:13.251609+0000","last_unstale":"2024-09-18T07:17:03.171591+0000","last_undegraded":"2024-09-18T07:17:03.171591+0000","last_fullsized":"2024-09-18T07:17:03.171591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:01.748551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040903899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"25'7878","reported_seq":7894,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:18:08.279680+0000","last_change":"2024-09-18T07:14:22.283241+0000","last_active":"2024-09-18T07:18:08.279680+0000","last_peered":"2024-09-18T07:18:08.279680+0000","last_clean":"2024-09-18T07:18:08.279680+0000","last_became_active":"2024-09-18T07:14:22.282926+0000","last_became_peered":"2024-09-18T07:14:22.282926+0000","last_unstale":"2024-09-18T07:18:08.279680+0000","last_undegraded":"2024-09-18T07:18:08.279680+0000","last_fullsized":"2024-09-18T07:18:08.279680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7878,"log_dups_size":0,"ondisk_log_size":7878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:11:58.849404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141926400,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7854,"num_write_kb":222187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138640+0000","last_change":"2024-09-18T07:14:15.249860+0000","last_active":"2024-09-18T07:17:03.138640+0000","last_peered":"2024-09-18T07:17:03.138640+0000","last_clean":"2024-09-18T07:17:03.138640+0000","last_became_active":"2024-09-18T07:14:13.251585+0000","last_became_peered":"2024-09-18T07:14:13.251585+0000","last_unstale":"2024-09-18T07:17:03.138640+0000","last_undegraded":"2024-09-18T07:17:03.138640+0000","last_fullsized":"2024-09-18T07:17:03.138640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:02:29.914389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039885799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"25'4220","reported_seq":4246,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.375214+0000","last_change":"2024-09-18T07:14:22.288077+0000","last_active":"2024-09-18T07:17:03.375214+0000","last_peered":"2024-09-18T07:17:03.375214+0000","last_clean":"2024-09-18T07:17:03.375214+0000","last_became_active":"2024-09-18T07:14:22.287843+0000","last_became_peered":"2024-09-18T07:14:22.287843+0000","last_unstale":"2024-09-18T07:17:03.375214+0000","last_undegraded":"2024-09-18T07:17:03.375214+0000","last_fullsized":"2024-09-18T07:17:03.375214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4220,"log_dups_size":0,"ondisk_log_size":4220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:13.075300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141164544,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":4202,"num_write_kb":175065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.148047+0000","last_change":"2024-09-18T07:14:15.603034+0000","last_active":"2024-09-18T07:17:03.148047+0000","last_peered":"2024-09-18T07:17:03.148047+0000","last_clean":"2024-09-18T07:17:03.148047+0000","last_became_active":"2024-09-18T07:14:13.957168+0000","last_became_peered":"2024-09-18T07:14:13.957168+0000","last_unstale":"2024-09-18T07:17:03.148047+0000","last_undegraded":"2024-09-18T07:17:03.148047+0000","last_fullsized":"2024-09-18T07:17:03.148047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:57:48.846502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048630800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"25'7512","reported_seq":7528,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:20.431132+0000","last_change":"2024-09-18T07:14:22.475409+0000","last_active":"2024-09-18T07:17:20.431132+0000","last_peered":"2024-09-18T07:17:20.431132+0000","last_clean":"2024-09-18T07:17:20.431132+0000","last_became_active":"2024-09-18T07:14:22.475086+0000","last_became_peered":"2024-09-18T07:14:22.475086+0000","last_unstale":"2024-09-18T07:17:20.431132+0000","last_undegraded":"2024-09-18T07:17:20.431132+0000","last_fullsized":"2024-09-18T07:17:20.431132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7512,"log_dups_size":0,"ondisk_log_size":7512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:33:15.601379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7506,"num_write_kb":189960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138588+0000","last_change":"2024-09-18T07:14:15.249863+0000","last_active":"2024-09-18T07:17:03.138588+0000","last_peered":"2024-09-18T07:17:03.138588+0000","last_clean":"2024-09-18T07:17:03.138588+0000","last_became_active":"2024-09-18T07:14:13.956929+0000","last_became_peered":"2024-09-18T07:14:13.956929+0000","last_unstale":"2024-09-18T07:17:03.138588+0000","last_undegraded":"2024-09-18T07:17:03.138588+0000","last_fullsized":"2024-09-18T07:17:03.138588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:11:55.607274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045339399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'5270","reported_seq":5286,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:22.633206+0000","last_change":"2024-09-18T07:14:22.288367+0000","last_active":"2024-09-18T07:17:22.633206+0000","last_peered":"2024-09-18T07:17:22.633206+0000","last_clean":"2024-09-18T07:17:22.633206+0000","last_became_active":"2024-09-18T07:14:22.288096+0000","last_became_peered":"2024-09-18T07:14:22.288096+0000","last_unstale":"2024-09-18T07:17:22.633206+0000","last_undegraded":"2024-09-18T07:17:22.633206+0000","last_fullsized":"2024-09-18T07:17:22.633206+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5270,"log_dups_size":0,"ondisk_log_size":5270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:39:40.638354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5267,"num_write_kb":201861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'3608","reported_seq":3634,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.140959+0000","last_change":"2024-09-18T07:14:22.284180+0000","last_active":"2024-09-18T07:17:03.140959+0000","last_peered":"2024-09-18T07:17:03.140959+0000","last_clean":"2024-09-18T07:17:03.140959+0000","last_became_active":"2024-09-18T07:14:22.283205+0000","last_became_peered":"2024-09-18T07:14:22.283205+0000","last_unstale":"2024-09-18T07:17:03.140959+0000","last_undegraded":"2024-09-18T07:17:03.140959+0000","last_fullsized":"2024-09-18T07:17:03.140959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3608,"log_dups_size":0,"ondisk_log_size":3608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:36:44.407465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92283408,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":37,"num_read_kb":29,"num_write":3631,"num_write_kb":171797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138107+0000","last_change":"2024-09-18T07:14:15.255874+0000","last_active":"2024-09-18T07:17:03.138107+0000","last_peered":"2024-09-18T07:17:03.138107+0000","last_clean":"2024-09-18T07:17:03.138107+0000","last_became_active":"2024-09-18T07:14:13.243579+0000","last_became_peered":"2024-09-18T07:14:13.243579+0000","last_unstale":"2024-09-18T07:17:03.138107+0000","last_undegraded":"2024-09-18T07:17:03.138107+0000","last_fullsized":"2024-09-18T07:17:03.138107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:28.722433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096575000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"25'4956","reported_seq":4980,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.371555+0000","last_change":"2024-09-18T07:14:22.475468+0000","last_active":"2024-09-18T07:17:03.371555+0000","last_peered":"2024-09-18T07:17:03.371555+0000","last_clean":"2024-09-18T07:17:03.371555+0000","last_became_active":"2024-09-18T07:14:22.474980+0000","last_became_peered":"2024-09-18T07:14:22.474980+0000","last_unstale":"2024-09-18T07:17:03.371555+0000","last_undegraded":"2024-09-18T07:17:03.371555+0000","last_fullsized":"2024-09-18T07:17:03.371555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4956,"log_dups_size":0,"ondisk_log_size":4956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:38:28.516038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124452864,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4927,"num_write_kb":170572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138277+0000","last_change":"2024-09-18T07:14:15.398157+0000","last_active":"2024-09-18T07:17:03.138277+0000","last_peered":"2024-09-18T07:17:03.138277+0000","last_clean":"2024-09-18T07:17:03.138277+0000","last_became_active":"2024-09-18T07:14:13.392213+0000","last_became_peered":"2024-09-18T07:14:13.392213+0000","last_unstale":"2024-09-18T07:17:03.138277+0000","last_undegraded":"2024-09-18T07:17:03.138277+0000","last_fullsized":"2024-09-18T07:17:03.138277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:47:04.187501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000312059,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"25'3057","reported_seq":3084,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.323804+0000","last_change":"2024-09-18T07:14:22.286270+0000","last_active":"2024-09-18T07:17:03.323804+0000","last_peered":"2024-09-18T07:17:03.323804+0000","last_clean":"2024-09-18T07:17:03.323804+0000","last_became_active":"2024-09-18T07:14:22.286007+0000","last_became_peered":"2024-09-18T07:14:22.286007+0000","last_unstale":"2024-09-18T07:17:03.323804+0000","last_undegraded":"2024-09-18T07:17:03.323804+0000","last_fullsized":"2024-09-18T07:17:03.323804+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3057,"log_dups_size":0,"ondisk_log_size":3057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:21:48.710903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92405760,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":3056,"num_write_kb":144258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.147637+0000","last_change":"2024-09-18T07:14:15.603018+0000","last_active":"2024-09-18T07:17:03.147637+0000","last_peered":"2024-09-18T07:17:03.147637+0000","last_clean":"2024-09-18T07:17:03.147637+0000","last_became_active":"2024-09-18T07:14:13.251213+0000","last_became_peered":"2024-09-18T07:14:13.251213+0000","last_unstale":"2024-09-18T07:17:03.147637+0000","last_undegraded":"2024-09-18T07:17:03.147637+0000","last_fullsized":"2024-09-18T07:17:03.147637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:43:22.952115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046971900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"25'5336","reported_seq":5357,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:15.364342+0000","last_change":"2024-09-18T07:14:22.284250+0000","last_active":"2024-09-18T07:17:15.364342+0000","last_peered":"2024-09-18T07:17:15.364342+0000","last_clean":"2024-09-18T07:17:15.364342+0000","last_became_active":"2024-09-18T07:14:22.283928+0000","last_became_peered":"2024-09-18T07:14:22.283928+0000","last_unstale":"2024-09-18T07:17:15.364342+0000","last_undegraded":"2024-09-18T07:17:15.364342+0000","last_fullsized":"2024-09-18T07:17:15.364342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:36:44.311383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080896,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":18,"num_read_kb":13,"num_write":5351,"num_write_kb":144518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138107+0000","last_change":"2024-09-18T07:14:15.255301+0000","last_active":"2024-09-18T07:17:03.138107+0000","last_peered":"2024-09-18T07:17:03.138107+0000","last_clean":"2024-09-18T07:17:03.138107+0000","last_became_active":"2024-09-18T07:14:13.243793+0000","last_became_peered":"2024-09-18T07:14:13.243793+0000","last_unstale":"2024-09-18T07:17:03.138107+0000","last_undegraded":"2024-09-18T07:17:03.138107+0000","last_fullsized":"2024-09-18T07:17:03.138107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:24:14.077910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039863000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":121,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.147737+0000","last_change":"2024-09-18T07:14:06.881800+0000","last_active":"2024-09-18T07:17:03.147737+0000","last_peered":"2024-09-18T07:17:03.147737+0000","last_clean":"2024-09-18T07:17:03.147737+0000","last_became_active":"2024-09-18T07:14:06.880771+0000","last_became_peered":"2024-09-18T07:14:06.880771+0000","last_unstale":"2024-09-18T07:17:03.147737+0000","last_undegraded":"2024-09-18T07:17:03.147737+0000","last_fullsized":"2024-09-18T07:17:03.147737+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:05.472923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:05.472923+0000","last_clean_scrub_stamp":"2024-09-18T07:14:05.472923+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:07.064297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"25'5233","reported_seq":5249,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.374017+0000","last_change":"2024-09-18T07:14:22.280027+0000","last_active":"2024-09-18T07:17:03.374017+0000","last_peered":"2024-09-18T07:17:03.374017+0000","last_clean":"2024-09-18T07:17:03.374017+0000","last_became_active":"2024-09-18T07:14:22.279712+0000","last_became_peered":"2024-09-18T07:14:22.279712+0000","last_unstale":"2024-09-18T07:17:03.374017+0000","last_undegraded":"2024-09-18T07:17:03.374017+0000","last_fullsized":"2024-09-18T07:17:03.374017+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:35:31.397370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136970240,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5162,"num_write_kb":248325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"25'8264","reported_seq":8400,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:19.307021+0000","last_change":"2024-09-18T07:14:22.475260+0000","last_active":"2024-09-18T07:17:19.307021+0000","last_peered":"2024-09-18T07:17:19.307021+0000","last_clean":"2024-09-18T07:17:19.307021+0000","last_became_active":"2024-09-18T07:14:22.475044+0000","last_became_peered":"2024-09-18T07:14:22.475044+0000","last_unstale":"2024-09-18T07:17:19.307021+0000","last_undegraded":"2024-09-18T07:17:19.307021+0000","last_fullsized":"2024-09-18T07:17:19.307021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":8264,"log_dups_size":0,"ondisk_log_size":8264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:52:46.307809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":384,"num_read_kb":402,"num_write":5199,"num_write_kb":145227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"25'6938","reported_seq":6956,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:20.283862+0000","last_change":"2024-09-18T07:14:22.286302+0000","last_active":"2024-09-18T07:17:20.283862+0000","last_peered":"2024-09-18T07:17:20.283862+0000","last_clean":"2024-09-18T07:17:20.283862+0000","last_became_active":"2024-09-18T07:14:22.285050+0000","last_became_peered":"2024-09-18T07:14:22.285050+0000","last_unstale":"2024-09-18T07:17:20.283862+0000","last_undegraded":"2024-09-18T07:17:20.283862+0000","last_fullsized":"2024-09-18T07:17:20.283862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:36:34.664394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146120704,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":6915,"num_write_kb":222193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"24'4813","reported_seq":4845,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.168379+0000","last_change":"2024-09-18T07:14:22.284686+0000","last_active":"2024-09-18T07:17:03.168379+0000","last_peered":"2024-09-18T07:17:03.168379+0000","last_clean":"2024-09-18T07:17:03.168379+0000","last_became_active":"2024-09-18T07:14:22.284397+0000","last_became_peered":"2024-09-18T07:14:22.284397+0000","last_unstale":"2024-09-18T07:17:03.168379+0000","last_undegraded":"2024-09-18T07:17:03.168379+0000","last_fullsized":"2024-09-18T07:17:03.168379+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4813,"log_dups_size":0,"ondisk_log_size":4813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:02:38.728384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104988672,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":64,"num_read_kb":50,"num_write":4839,"num_write_kb":163526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"25'4924","reported_seq":4951,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.373267+0000","last_change":"2024-09-18T07:14:22.288110+0000","last_active":"2024-09-18T07:17:03.373267+0000","last_peered":"2024-09-18T07:17:03.373267+0000","last_clean":"2024-09-18T07:17:03.373267+0000","last_became_active":"2024-09-18T07:14:22.287513+0000","last_became_peered":"2024-09-18T07:14:22.287513+0000","last_unstale":"2024-09-18T07:17:03.373267+0000","last_undegraded":"2024-09-18T07:17:03.373267+0000","last_fullsized":"2024-09-18T07:17:03.373267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4924,"log_dups_size":0,"ondisk_log_size":4924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:02:43.799881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794368,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":43,"num_read_kb":96,"num_write":4998,"num_write_kb":181316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"25'3047","reported_seq":3065,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.318954+0000","last_change":"2024-09-18T07:14:22.289036+0000","last_active":"2024-09-18T07:17:03.318954+0000","last_peered":"2024-09-18T07:17:03.318954+0000","last_clean":"2024-09-18T07:17:03.318954+0000","last_became_active":"2024-09-18T07:14:22.288690+0000","last_became_peered":"2024-09-18T07:14:22.288690+0000","last_unstale":"2024-09-18T07:17:03.318954+0000","last_undegraded":"2024-09-18T07:17:03.318954+0000","last_fullsized":"2024-09-18T07:17:03.318954+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3047,"log_dups_size":0,"ondisk_log_size":3047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:02:01.547710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92275200,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":3030,"num_write_kb":128003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"25'3892","reported_seq":3976,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.404529+0000","last_change":"2024-09-18T07:14:22.289068+0000","last_active":"2024-09-18T07:17:03.404529+0000","last_peered":"2024-09-18T07:17:03.404529+0000","last_clean":"2024-09-18T07:17:03.404529+0000","last_became_active":"2024-09-18T07:14:22.288767+0000","last_became_peered":"2024-09-18T07:14:22.288767+0000","last_unstale":"2024-09-18T07:17:03.404529+0000","last_undegraded":"2024-09-18T07:17:03.404529+0000","last_fullsized":"2024-09-18T07:17:03.404529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:08:25.598857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106375352,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":259,"num_read_kb":194,"num_write":4137,"num_write_kb":154845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"25'5118","reported_seq":5134,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:18:08.277515+0000","last_change":"2024-09-18T07:14:22.289787+0000","last_active":"2024-09-18T07:18:08.277515+0000","last_peered":"2024-09-18T07:18:08.277515+0000","last_clean":"2024-09-18T07:18:08.277515+0000","last_became_active":"2024-09-18T07:14:22.289527+0000","last_became_peered":"2024-09-18T07:14:22.289527+0000","last_unstale":"2024-09-18T07:18:08.277515+0000","last_undegraded":"2024-09-18T07:18:08.277515+0000","last_fullsized":"2024-09-18T07:18:08.277515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5118,"log_dups_size":0,"ondisk_log_size":5118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:39:43.412358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130834432,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5068,"num_write_kb":190489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"25'3027","reported_seq":3348,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.372468+0000","last_change":"2024-09-18T07:14:22.285350+0000","last_active":"2024-09-18T07:17:03.372468+0000","last_peered":"2024-09-18T07:17:03.372468+0000","last_clean":"2024-09-18T07:17:03.372468+0000","last_became_active":"2024-09-18T07:14:22.284989+0000","last_became_peered":"2024-09-18T07:14:22.284989+0000","last_unstale":"2024-09-18T07:17:03.372468+0000","last_undegraded":"2024-09-18T07:17:03.372468+0000","last_fullsized":"2024-09-18T07:17:03.372468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3027,"log_dups_size":0,"ondisk_log_size":3027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:58:31.394769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71434240,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1022,"num_read_kb":916,"num_write":3192,"num_write_kb":144514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"24'3065","reported_seq":3104,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.147623+0000","last_change":"2024-09-18T07:14:22.290602+0000","last_active":"2024-09-18T07:17:03.147623+0000","last_peered":"2024-09-18T07:17:03.147623+0000","last_clean":"2024-09-18T07:17:03.147623+0000","last_became_active":"2024-09-18T07:14:22.290299+0000","last_became_peered":"2024-09-18T07:14:22.290299+0000","last_unstale":"2024-09-18T07:17:03.147623+0000","last_undegraded":"2024-09-18T07:17:03.147623+0000","last_fullsized":"2024-09-18T07:17:03.147623+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3065,"log_dups_size":0,"ondisk_log_size":3065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:31:47.964089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":3095,"num_write_kb":160450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"25'3705","reported_seq":3721,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.374545+0000","last_change":"2024-09-18T07:14:22.290568+0000","last_active":"2024-09-18T07:17:03.374545+0000","last_peered":"2024-09-18T07:17:03.374545+0000","last_clean":"2024-09-18T07:17:03.374545+0000","last_became_active":"2024-09-18T07:14:22.290308+0000","last_became_peered":"2024-09-18T07:14:22.290308+0000","last_unstale":"2024-09-18T07:17:03.374545+0000","last_undegraded":"2024-09-18T07:17:03.374545+0000","last_fullsized":"2024-09-18T07:17:03.374545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3705,"log_dups_size":0,"ondisk_log_size":3705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:17.589485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120979456,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3652,"num_write_kb":176387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"25'7744","reported_seq":7771,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:13.571757+0000","last_change":"2024-09-18T07:14:22.288236+0000","last_active":"2024-09-18T07:17:13.571757+0000","last_peered":"2024-09-18T07:17:13.571757+0000","last_clean":"2024-09-18T07:17:13.571757+0000","last_became_active":"2024-09-18T07:14:22.287927+0000","last_became_peered":"2024-09-18T07:14:22.287927+0000","last_unstale":"2024-09-18T07:17:13.571757+0000","last_undegraded":"2024-09-18T07:17:13.571757+0000","last_fullsized":"2024-09-18T07:17:13.571757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7744,"log_dups_size":0,"ondisk_log_size":7744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:26:51.355433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":31,"num_read_kb":87,"num_write":7741,"num_write_kb":165450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3581707016,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":2509,"num_read_kb":2520,"num_write":165325,"num_write_kb":5659033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6865756160,"data_stored":6861482136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":168164,"ondisk_log_size":168164,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1309716,"kb_used_data":1043972,"kb_used_omap":1,"kb_used_meta":265726,"kb_avail":93062124,"statfs":{"total":96636764160,"available":95295614976,"internally_reserved":0,"allocated":1069027328,"data_stored":1068306725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":272103882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408832,"kb_used_data":311024,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93963008,"statfs":{"total":96636764160,"available":96218120192,"internally_reserved":0,"allocated":318488576,"data_stored":318129409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":100137414},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542196,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1674096,"kb_used_data":1300128,"kb_used_omap":1,"kb_used_meta":373950,"kb_avail":92697744,"statfs":{"total":96636764160,"available":94922489856,"internally_reserved":0,"allocated":1331331072,"data_stored":1330396577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":382925253},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542196,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1384272,"kb_used_data":1082432,"kb_used_omap":1,"kb_used_meta":301822,"kb_avail":92987568,"statfs":{"total":96636764160,"available":95219269632,"internally_reserved":0,"allocated":1108410368,"data_stored":1107550749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":309066183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542196,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436740,"kb_used_data":338932,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93935100,"statfs":{"total":96636764160,"available":96189542400,"internally_reserved":0,"allocated":347066368,"data_stored":346699873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":100137413},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542196,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902300,"kb_used_data":676812,"kb_used_omap":1,"kb_used_meta":225470,"kb_avail":93469540,"statfs":{"total":96636764160,"available":95712808960,"internally_reserved":0,"allocated":693055488,"data_stored":692411364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":230881733},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542196,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1276088,"kb_used_data":1002024,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":93095752,"statfs":{"total":96636764160,"available":95330050048,"internally_reserved":0,"allocated":1026072576,"data_stored":1025248181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":280623557},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1182036,"kb_used_data":952388,"kb_used_omap":1,"kb_used_meta":229630,"kb_avail":93189804,"statfs":{"total":96636764160,"available":95426359296,"internally_reserved":0,"allocated":975245312,"data_stored":974453512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235141573},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1025306624,"data_stored":1024637208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":692817920,"data_stored":692329300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346832896,"data_stored":346617828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1108176896,"data_stored":1107468704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1331097600,"data_stored":1330314532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":975007744,"data_stored":974371448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":318255104,"data_stored":318047364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1068261376,"data_stored":1067695752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T07:18:20.122 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T07:18:20.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.216+0000 7f562fc16640 1 Processor -- start 2024-09-18T07:18:20.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.216+0000 7f562fc16640 1 -- start start 2024-09-18T07:18:20.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.217+0000 7f562fc16640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5628074fe0 con 0x7f5628142db0 2024-09-18T07:18:20.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.217+0000 7f562fc16640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5628073f60 con 0x7f5628147350 2024-09-18T07:18:20.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.217+0000 7f562fc16640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5628144b70 con 0x7f5628074320 2024-09-18T07:18:20.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.217+0000 7f562d1b1640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f5628074320 0x7f5628147730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:36412/0 (socket says 172.21.15.70:36412) 2024-09-18T07:18:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.217+0000 7f562e1b3640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5628142db0 0x7f5628143240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34138/0 (socket says 172.21.15.70:34138) 2024-09-18T07:18:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.217+0000 7f562d1b1640 1 -- 172.21.15.70:0/2277660606 learned_addr learned my addr 172.21.15.70:0/2277660606 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.217+0000 7f562c9b0640 1 -- 172.21.15.70:0/2277660606 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3933221146 0 0) 0x7f5628074fe0 con 0x7f5628142db0 2024-09-18T07:18:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.218+0000 7f562c9b0640 1 -- 172.21.15.70:0/2277660606 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5614003520 con 0x7f5628142db0 2024-09-18T07:18:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.218+0000 7f562c9b0640 1 -- 172.21.15.70:0/2277660606 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1707685344 0 0) 0x7f5628144b70 con 0x7f5628074320 2024-09-18T07:18:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.218+0000 7f562c9b0640 1 -- 172.21.15.70:0/2277660606 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5628074fe0 con 0x7f5628074320 2024-09-18T07:18:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.218+0000 7f562c9b0640 1 -- 172.21.15.70:0/2277660606 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1212349563 0 0) 0x7f5614003520 con 0x7f5628142db0 2024-09-18T07:18:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.218+0000 7f562c9b0640 1 -- 172.21.15.70:0/2277660606 >> v1:172.21.15.70:6790/0 conn(0x7f5628074320 legacy=0x7f5628147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.218+0000 7f562c9b0640 1 -- 172.21.15.70:0/2277660606 >> v1:172.21.15.172:6789/0 conn(0x7f5628147350 legacy=0x7f5628071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.218+0000 7f562c9b0640 1 -- 172.21.15.70:0/2277660606 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5628147e40 con 0x7f5628142db0 2024-09-18T07:18:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.218+0000 7f562c9b0640 1 -- 172.21.15.70:0/2277660606 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f5620001690 con 0x7f5628142db0 2024-09-18T07:18:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.218+0000 7f562c9b0640 1 -- 172.21.15.70:0/2277660606 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5620001830 con 0x7f5628142db0 2024-09-18T07:18:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.218+0000 7f562c9b0640 1 -- 172.21.15.70:0/2277660606 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f5620001cc0 con 0x7f5628142db0 2024-09-18T07:18:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.218+0000 7f562fc16640 1 -- 172.21.15.70:0/2277660606 >> v1:172.21.15.70:6789/0 conn(0x7f5628142db0 legacy=0x7f5628143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.218+0000 7f562fc16640 1 -- 172.21.15.70:0/2277660606 shutdown_connections 2024-09-18T07:18:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.218+0000 7f562fc16640 1 -- 172.21.15.70:0/2277660606 >> 172.21.15.70:0/2277660606 conn(0x7f56280b64d0 msgr2=0x7f56280b68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.219+0000 7f562fc16640 1 -- 172.21.15.70:0/2277660606 shutdown_connections 2024-09-18T07:18:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.219+0000 7f562fc16640 1 -- 172.21.15.70:0/2277660606 wait complete. 2024-09-18T07:18:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.219+0000 7f562fc16640 1 Processor -- start 2024-09-18T07:18:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.220+0000 7f562fc16640 1 -- start start 2024-09-18T07:18:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.220+0000 7f562fc16640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5628137ad0 con 0x7f5628142db0 2024-09-18T07:18:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.220+0000 7f562fc16640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56281d9af0 con 0x7f5628147350 2024-09-18T07:18:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.220+0000 7f562fc16640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56281dccd0 con 0x7f5628074320 2024-09-18T07:18:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.220+0000 7f562d9b2640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f5628074320 0x7f5628137ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:36426/0 (socket says 172.21.15.70:36426) 2024-09-18T07:18:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.220+0000 7f562d1b1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5628142db0 0x7f5628132c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34148/0 (socket says 172.21.15.70:34148) 2024-09-18T07:18:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.220+0000 7f562d9b2640 1 -- 172.21.15.70:0/1676644989 learned_addr learned my addr 172.21.15.70:0/1676644989 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.220+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1983452691 0 0) 0x7f5628137ad0 con 0x7f5628142db0 2024-09-18T07:18:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.220+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f560c003520 con 0x7f5628142db0 2024-09-18T07:18:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.220+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 775429450 0 0) 0x7f56281dccd0 con 0x7f5628074320 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.220+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5628137ad0 con 0x7f5628074320 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2148967309 0 0) 0x7f56281d9af0 con 0x7f5628147350 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56281dccd0 con 0x7f5628147350 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2410971202 0 0) 0x7f5628137ad0 con 0x7f5628074320 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f56281d9af0 con 0x7f5628074320 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 653107542 0 0) 0x7f560c003520 con 0x7f5628142db0 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5628137ad0 con 0x7f5628142db0 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f5624001c90 con 0x7f5628074320 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f56180011c0 con 0x7f5628142db0 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4011698461 0 0) 0x7f56281dccd0 con 0x7f5628147350 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f560c003520 con 0x7f5628147350 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f5620001b80 con 0x7f5628147350 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2447916189 0 0) 0x7f56281d9af0 con 0x7f5628074320 2024-09-18T07:18:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 >> v1:172.21.15.172:6789/0 conn(0x7f5628147350 legacy=0x7f56281e4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 >> v1:172.21.15.70:6789/0 conn(0x7f5628142db0 legacy=0x7f5628132c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56281dcea0 con 0x7f5628074320 2024-09-18T07:18:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f562fc16640 1 -- 172.21.15.70:0/1676644989 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5628138700 con 0x7f5628074320 2024-09-18T07:18:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f562fc16640 1 -- 172.21.15.70:0/1676644989 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f5628133500 con 0x7f5628074320 2024-09-18T07:18:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5624001fd0 con 0x7f5628074320 2024-09-18T07:18:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.221+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f5624003de0 con 0x7f5628074320 2024-09-18T07:18:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.222+0000 7f562fc16640 1 -- 172.21.15.70:0/1676644989 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5610005080 con 0x7f5628074320 2024-09-18T07:18:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.222+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f5624003de0 con 0x7f5628074320 2024-09-18T07:18:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.223+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f5624096380 con 0x7f5628074320 2024-09-18T07:18:20.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.227+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f562405eb80 con 0x7f5628074320 2024-09-18T07:18:20.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.403+0000 7f562fc16640 1 -- 172.21.15.70:0/1676644989 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5610000b90 con 0x7f560c079930 2024-09-18T07:18:20.408 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.406+0000 7f561e7fc640 1 -- 172.21.15.70:0/1676644989 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117442 (unknown 2965378022 0 272469850) 0x7f5610000b90 con 0x7f560c079930 2024-09-18T07:18:20.409 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:18:20.410 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-18T07:18:20.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.408+0000 7f562fc16640 1 -- 172.21.15.70:0/1676644989 >> v1:172.21.15.70:6816/1657990791 conn(0x7f560c079930 legacy=0x7f560c07bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.408+0000 7f562fc16640 1 -- 172.21.15.70:0/1676644989 >> v1:172.21.15.70:6790/0 conn(0x7f5628074320 legacy=0x7f5628137ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.408+0000 7f562fc16640 1 -- 172.21.15.70:0/1676644989 shutdown_connections 2024-09-18T07:18:20.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.408+0000 7f562fc16640 1 -- 172.21.15.70:0/1676644989 >> 172.21.15.70:0/1676644989 conn(0x7f56280b64d0 msgr2=0x7f5628143ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:20.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.408+0000 7f562fc16640 1 -- 172.21.15.70:0/1676644989 shutdown_connections 2024-09-18T07:18:20.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.409+0000 7f562fc16640 1 -- 172.21.15.70:0/1676644989 wait complete. 2024-09-18T07:18:20.423 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-18T07:18:19.496357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3582297403,"num_objects":886,"num_object_clones":0,"num_object_copies":1772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":886,"num_whiteouts":0,"num_read":2575,"num_read_kb":2575,"num_write":165428,"num_write_kb":5660663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":168239,"ondisk_log_size":168239,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8574080,"kb_used_data":6707712,"kb_used_omap":12,"kb_used_meta":1866227,"kb_avail":746400640,"statfs":{"total":773094113280,"available":764314255360,"internally_reserved":0,"allocated":6868697088,"data_stored":6863196390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1911017008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":69632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002343"},"pg_stats":[{"pgid":"3.1f","version":"24'3617","reported_seq":3635,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.171919+0000","last_change":"2024-09-18T07:14:22.286661+0000","last_active":"2024-09-18T07:17:03.171919+0000","last_peered":"2024-09-18T07:17:03.171919+0000","last_clean":"2024-09-18T07:17:03.171919+0000","last_became_active":"2024-09-18T07:14:22.286401+0000","last_became_peered":"2024-09-18T07:14:22.286401+0000","last_unstale":"2024-09-18T07:17:03.171919+0000","last_undegraded":"2024-09-18T07:17:03.171919+0000","last_fullsized":"2024-09-18T07:17:03.171919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3617,"log_dups_size":0,"ondisk_log_size":3617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:26:38.664961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95160672,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":3623,"num_write_kb":151525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"25'6414","reported_seq":6531,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:09.318453+0000","last_change":"2024-09-18T07:14:22.284740+0000","last_active":"2024-09-18T07:17:09.318453+0000","last_peered":"2024-09-18T07:17:09.318453+0000","last_clean":"2024-09-18T07:17:09.318453+0000","last_became_active":"2024-09-18T07:14:22.284487+0000","last_became_peered":"2024-09-18T07:14:22.284487+0000","last_unstale":"2024-09-18T07:17:09.318453+0000","last_undegraded":"2024-09-18T07:17:09.318453+0000","last_fullsized":"2024-09-18T07:17:09.318453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":6414,"log_dups_size":0,"ondisk_log_size":6414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:23:39.072202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377808,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":6397,"num_write_kb":190429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"25'7587","reported_seq":7636,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:14.357682+0000","last_change":"2024-09-18T07:14:22.288902+0000","last_active":"2024-09-18T07:17:14.357682+0000","last_peered":"2024-09-18T07:17:14.357682+0000","last_clean":"2024-09-18T07:17:14.357682+0000","last_became_active":"2024-09-18T07:14:22.288441+0000","last_became_peered":"2024-09-18T07:14:22.288441+0000","last_unstale":"2024-09-18T07:17:14.357682+0000","last_undegraded":"2024-09-18T07:17:14.357682+0000","last_fullsized":"2024-09-18T07:17:14.357682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7587,"log_dups_size":0,"ondisk_log_size":7587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:04:30.016419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167903232,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":127,"num_read_kb":103,"num_write":7620,"num_write_kb":243849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"24'3255","reported_seq":3290,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138103+0000","last_change":"2024-09-18T07:14:22.288842+0000","last_active":"2024-09-18T07:17:03.138103+0000","last_peered":"2024-09-18T07:17:03.138103+0000","last_clean":"2024-09-18T07:17:03.138103+0000","last_became_active":"2024-09-18T07:14:22.287853+0000","last_became_peered":"2024-09-18T07:14:22.287853+0000","last_unstale":"2024-09-18T07:17:03.138103+0000","last_undegraded":"2024-09-18T07:17:03.138103+0000","last_fullsized":"2024-09-18T07:17:03.138103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3255,"log_dups_size":0,"ondisk_log_size":3255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:14.436120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88211456,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":79,"num_read_kb":61,"num_write":3296,"num_write_kb":156739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"25'3829","reported_seq":3848,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:18:08.280825+0000","last_change":"2024-09-18T07:14:22.286284+0000","last_active":"2024-09-18T07:18:08.280825+0000","last_peered":"2024-09-18T07:18:08.280825+0000","last_clean":"2024-09-18T07:18:08.280825+0000","last_became_active":"2024-09-18T07:14:22.286018+0000","last_became_peered":"2024-09-18T07:14:22.286018+0000","last_unstale":"2024-09-18T07:18:08.280825+0000","last_undegraded":"2024-09-18T07:18:08.280825+0000","last_fullsized":"2024-09-18T07:18:08.280825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3829,"log_dups_size":0,"ondisk_log_size":3829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:03:53.111506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91650400,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":3788,"num_write_kb":184441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"25'5565","reported_seq":5585,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:07.526513+0000","last_change":"2024-09-18T07:14:22.280275+0000","last_active":"2024-09-18T07:17:07.526513+0000","last_peered":"2024-09-18T07:17:07.526513+0000","last_clean":"2024-09-18T07:17:07.526513+0000","last_became_active":"2024-09-18T07:14:22.279826+0000","last_became_peered":"2024-09-18T07:14:22.279826+0000","last_unstale":"2024-09-18T07:17:07.526513+0000","last_undegraded":"2024-09-18T07:17:07.526513+0000","last_fullsized":"2024-09-18T07:17:07.526513+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5565,"log_dups_size":0,"ondisk_log_size":5565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:19:40.864770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":15,"num_read_kb":75,"num_write":5550,"num_write_kb":155179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"25'4743","reported_seq":4775,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:18.223911+0000","last_change":"2024-09-18T07:14:22.280164+0000","last_active":"2024-09-18T07:17:18.223911+0000","last_peered":"2024-09-18T07:17:18.223911+0000","last_clean":"2024-09-18T07:17:18.223911+0000","last_became_active":"2024-09-18T07:14:22.279885+0000","last_became_peered":"2024-09-18T07:14:22.279885+0000","last_unstale":"2024-09-18T07:17:18.223911+0000","last_undegraded":"2024-09-18T07:17:18.223911+0000","last_fullsized":"2024-09-18T07:17:18.223911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:08:22.395662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":4713,"num_write_kb":147181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"25'8007","reported_seq":8024,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:18.366347+0000","last_change":"2024-09-18T07:14:22.283501+0000","last_active":"2024-09-18T07:17:18.366347+0000","last_peered":"2024-09-18T07:17:18.366347+0000","last_clean":"2024-09-18T07:17:18.366347+0000","last_became_active":"2024-09-18T07:14:22.283231+0000","last_became_peered":"2024-09-18T07:14:22.283231+0000","last_unstale":"2024-09-18T07:17:18.366347+0000","last_undegraded":"2024-09-18T07:17:18.366347+0000","last_fullsized":"2024-09-18T07:17:18.366347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":8007,"log_dups_size":0,"ondisk_log_size":8007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:49:47.239612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125895176,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":8013,"num_write_kb":214525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"25'7235","reported_seq":7255,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:21.527844+0000","last_change":"2024-09-18T07:14:22.286153+0000","last_active":"2024-09-18T07:17:21.527844+0000","last_peered":"2024-09-18T07:17:21.527844+0000","last_clean":"2024-09-18T07:17:21.527844+0000","last_became_active":"2024-09-18T07:14:22.285884+0000","last_became_peered":"2024-09-18T07:14:22.285884+0000","last_unstale":"2024-09-18T07:17:21.527844+0000","last_undegraded":"2024-09-18T07:17:21.527844+0000","last_fullsized":"2024-09-18T07:17:21.527844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7235,"log_dups_size":0,"ondisk_log_size":7235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:36:41.719570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135659520,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":7238,"num_write_kb":200827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"25'3408","reported_seq":3432,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.319245+0000","last_change":"2024-09-18T07:14:22.289679+0000","last_active":"2024-09-18T07:17:03.319245+0000","last_peered":"2024-09-18T07:17:03.319245+0000","last_clean":"2024-09-18T07:17:03.319245+0000","last_became_active":"2024-09-18T07:14:22.289372+0000","last_became_peered":"2024-09-18T07:14:22.289372+0000","last_unstale":"2024-09-18T07:17:03.319245+0000","last_undegraded":"2024-09-18T07:17:03.319245+0000","last_fullsized":"2024-09-18T07:17:03.319245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3408,"log_dups_size":0,"ondisk_log_size":3408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:59:31.222383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112003944,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":22,"num_read_kb":143,"num_write":3412,"num_write_kb":151531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"25'8025","reported_seq":8044,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:17.126686+0000","last_change":"2024-09-18T07:14:22.288942+0000","last_active":"2024-09-18T07:17:17.126686+0000","last_peered":"2024-09-18T07:17:17.126686+0000","last_clean":"2024-09-18T07:17:17.126686+0000","last_became_active":"2024-09-18T07:14:22.288634+0000","last_became_peered":"2024-09-18T07:14:22.288634+0000","last_unstale":"2024-09-18T07:17:17.126686+0000","last_undegraded":"2024-09-18T07:17:17.126686+0000","last_fullsized":"2024-09-18T07:17:17.126686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":8025,"log_dups_size":0,"ondisk_log_size":8025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:34:15.636211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8027,"num_write_kb":201731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"25'4872","reported_seq":4889,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:10.976186+0000","last_change":"2024-09-18T07:14:22.291854+0000","last_active":"2024-09-18T07:17:10.976186+0000","last_peered":"2024-09-18T07:17:10.976186+0000","last_clean":"2024-09-18T07:17:10.976186+0000","last_became_active":"2024-09-18T07:14:22.291551+0000","last_became_peered":"2024-09-18T07:14:22.291551+0000","last_unstale":"2024-09-18T07:17:10.976186+0000","last_undegraded":"2024-09-18T07:17:10.976186+0000","last_fullsized":"2024-09-18T07:17:10.976186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4872,"log_dups_size":0,"ondisk_log_size":4872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:54:13.729656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303680,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4826,"num_write_kb":160133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.171591+0000","last_change":"2024-09-18T07:14:15.808642+0000","last_active":"2024-09-18T07:17:03.171591+0000","last_peered":"2024-09-18T07:17:03.171591+0000","last_clean":"2024-09-18T07:17:03.171591+0000","last_became_active":"2024-09-18T07:14:13.251609+0000","last_became_peered":"2024-09-18T07:14:13.251609+0000","last_unstale":"2024-09-18T07:17:03.171591+0000","last_undegraded":"2024-09-18T07:17:03.171591+0000","last_fullsized":"2024-09-18T07:17:03.171591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:01.748551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040903899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"25'7878","reported_seq":7894,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:18:08.279680+0000","last_change":"2024-09-18T07:14:22.283241+0000","last_active":"2024-09-18T07:18:08.279680+0000","last_peered":"2024-09-18T07:18:08.279680+0000","last_clean":"2024-09-18T07:18:08.279680+0000","last_became_active":"2024-09-18T07:14:22.282926+0000","last_became_peered":"2024-09-18T07:14:22.282926+0000","last_unstale":"2024-09-18T07:18:08.279680+0000","last_undegraded":"2024-09-18T07:18:08.279680+0000","last_fullsized":"2024-09-18T07:18:08.279680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7878,"log_dups_size":0,"ondisk_log_size":7878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:11:58.849404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141926400,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7854,"num_write_kb":222187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138640+0000","last_change":"2024-09-18T07:14:15.249860+0000","last_active":"2024-09-18T07:17:03.138640+0000","last_peered":"2024-09-18T07:17:03.138640+0000","last_clean":"2024-09-18T07:17:03.138640+0000","last_became_active":"2024-09-18T07:14:13.251585+0000","last_became_peered":"2024-09-18T07:14:13.251585+0000","last_unstale":"2024-09-18T07:17:03.138640+0000","last_undegraded":"2024-09-18T07:17:03.138640+0000","last_fullsized":"2024-09-18T07:17:03.138640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:02:29.914389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039885799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"25'4220","reported_seq":4246,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.375214+0000","last_change":"2024-09-18T07:14:22.288077+0000","last_active":"2024-09-18T07:17:03.375214+0000","last_peered":"2024-09-18T07:17:03.375214+0000","last_clean":"2024-09-18T07:17:03.375214+0000","last_became_active":"2024-09-18T07:14:22.287843+0000","last_became_peered":"2024-09-18T07:14:22.287843+0000","last_unstale":"2024-09-18T07:17:03.375214+0000","last_undegraded":"2024-09-18T07:17:03.375214+0000","last_fullsized":"2024-09-18T07:17:03.375214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4220,"log_dups_size":0,"ondisk_log_size":4220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:13.075300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141164544,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":4202,"num_write_kb":175065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.148047+0000","last_change":"2024-09-18T07:14:15.603034+0000","last_active":"2024-09-18T07:17:03.148047+0000","last_peered":"2024-09-18T07:17:03.148047+0000","last_clean":"2024-09-18T07:17:03.148047+0000","last_became_active":"2024-09-18T07:14:13.957168+0000","last_became_peered":"2024-09-18T07:14:13.957168+0000","last_unstale":"2024-09-18T07:17:03.148047+0000","last_undegraded":"2024-09-18T07:17:03.148047+0000","last_fullsized":"2024-09-18T07:17:03.148047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:57:48.846502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048630800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"25'7512","reported_seq":7528,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:20.431132+0000","last_change":"2024-09-18T07:14:22.475409+0000","last_active":"2024-09-18T07:17:20.431132+0000","last_peered":"2024-09-18T07:17:20.431132+0000","last_clean":"2024-09-18T07:17:20.431132+0000","last_became_active":"2024-09-18T07:14:22.475086+0000","last_became_peered":"2024-09-18T07:14:22.475086+0000","last_unstale":"2024-09-18T07:17:20.431132+0000","last_undegraded":"2024-09-18T07:17:20.431132+0000","last_fullsized":"2024-09-18T07:17:20.431132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7512,"log_dups_size":0,"ondisk_log_size":7512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:33:15.601379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7506,"num_write_kb":189960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138588+0000","last_change":"2024-09-18T07:14:15.249863+0000","last_active":"2024-09-18T07:17:03.138588+0000","last_peered":"2024-09-18T07:17:03.138588+0000","last_clean":"2024-09-18T07:17:03.138588+0000","last_became_active":"2024-09-18T07:14:13.956929+0000","last_became_peered":"2024-09-18T07:14:13.956929+0000","last_unstale":"2024-09-18T07:17:03.138588+0000","last_undegraded":"2024-09-18T07:17:03.138588+0000","last_fullsized":"2024-09-18T07:17:03.138588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:11:55.607274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045339399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'5270","reported_seq":5286,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:22.633206+0000","last_change":"2024-09-18T07:14:22.288367+0000","last_active":"2024-09-18T07:17:22.633206+0000","last_peered":"2024-09-18T07:17:22.633206+0000","last_clean":"2024-09-18T07:17:22.633206+0000","last_became_active":"2024-09-18T07:14:22.288096+0000","last_became_peered":"2024-09-18T07:14:22.288096+0000","last_unstale":"2024-09-18T07:17:22.633206+0000","last_undegraded":"2024-09-18T07:17:22.633206+0000","last_fullsized":"2024-09-18T07:17:22.633206+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5270,"log_dups_size":0,"ondisk_log_size":5270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:39:40.638354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5267,"num_write_kb":201861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'3608","reported_seq":3634,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.140959+0000","last_change":"2024-09-18T07:14:22.284180+0000","last_active":"2024-09-18T07:17:03.140959+0000","last_peered":"2024-09-18T07:17:03.140959+0000","last_clean":"2024-09-18T07:17:03.140959+0000","last_became_active":"2024-09-18T07:14:22.283205+0000","last_became_peered":"2024-09-18T07:14:22.283205+0000","last_unstale":"2024-09-18T07:17:03.140959+0000","last_undegraded":"2024-09-18T07:17:03.140959+0000","last_fullsized":"2024-09-18T07:17:03.140959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3608,"log_dups_size":0,"ondisk_log_size":3608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:36:44.407465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92283408,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":37,"num_read_kb":29,"num_write":3631,"num_write_kb":171797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138107+0000","last_change":"2024-09-18T07:14:15.255874+0000","last_active":"2024-09-18T07:17:03.138107+0000","last_peered":"2024-09-18T07:17:03.138107+0000","last_clean":"2024-09-18T07:17:03.138107+0000","last_became_active":"2024-09-18T07:14:13.243579+0000","last_became_peered":"2024-09-18T07:14:13.243579+0000","last_unstale":"2024-09-18T07:17:03.138107+0000","last_undegraded":"2024-09-18T07:17:03.138107+0000","last_fullsized":"2024-09-18T07:17:03.138107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:28.722433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096575000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"25'4956","reported_seq":4980,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.371555+0000","last_change":"2024-09-18T07:14:22.475468+0000","last_active":"2024-09-18T07:17:03.371555+0000","last_peered":"2024-09-18T07:17:03.371555+0000","last_clean":"2024-09-18T07:17:03.371555+0000","last_became_active":"2024-09-18T07:14:22.474980+0000","last_became_peered":"2024-09-18T07:14:22.474980+0000","last_unstale":"2024-09-18T07:17:03.371555+0000","last_undegraded":"2024-09-18T07:17:03.371555+0000","last_fullsized":"2024-09-18T07:17:03.371555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4956,"log_dups_size":0,"ondisk_log_size":4956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:38:28.516038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124452864,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4927,"num_write_kb":170572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138277+0000","last_change":"2024-09-18T07:14:15.398157+0000","last_active":"2024-09-18T07:17:03.138277+0000","last_peered":"2024-09-18T07:17:03.138277+0000","last_clean":"2024-09-18T07:17:03.138277+0000","last_became_active":"2024-09-18T07:14:13.392213+0000","last_became_peered":"2024-09-18T07:14:13.392213+0000","last_unstale":"2024-09-18T07:17:03.138277+0000","last_undegraded":"2024-09-18T07:17:03.138277+0000","last_fullsized":"2024-09-18T07:17:03.138277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:47:04.187501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000312059,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"25'3057","reported_seq":3084,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.323804+0000","last_change":"2024-09-18T07:14:22.286270+0000","last_active":"2024-09-18T07:17:03.323804+0000","last_peered":"2024-09-18T07:17:03.323804+0000","last_clean":"2024-09-18T07:17:03.323804+0000","last_became_active":"2024-09-18T07:14:22.286007+0000","last_became_peered":"2024-09-18T07:14:22.286007+0000","last_unstale":"2024-09-18T07:17:03.323804+0000","last_undegraded":"2024-09-18T07:17:03.323804+0000","last_fullsized":"2024-09-18T07:17:03.323804+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3057,"log_dups_size":0,"ondisk_log_size":3057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:21:48.710903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92405760,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":3056,"num_write_kb":144258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.147637+0000","last_change":"2024-09-18T07:14:15.603018+0000","last_active":"2024-09-18T07:17:03.147637+0000","last_peered":"2024-09-18T07:17:03.147637+0000","last_clean":"2024-09-18T07:17:03.147637+0000","last_became_active":"2024-09-18T07:14:13.251213+0000","last_became_peered":"2024-09-18T07:14:13.251213+0000","last_unstale":"2024-09-18T07:17:03.147637+0000","last_undegraded":"2024-09-18T07:17:03.147637+0000","last_fullsized":"2024-09-18T07:17:03.147637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:43:22.952115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046971900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"25'5336","reported_seq":5357,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:15.364342+0000","last_change":"2024-09-18T07:14:22.284250+0000","last_active":"2024-09-18T07:17:15.364342+0000","last_peered":"2024-09-18T07:17:15.364342+0000","last_clean":"2024-09-18T07:17:15.364342+0000","last_became_active":"2024-09-18T07:14:22.283928+0000","last_became_peered":"2024-09-18T07:14:22.283928+0000","last_unstale":"2024-09-18T07:17:15.364342+0000","last_undegraded":"2024-09-18T07:17:15.364342+0000","last_fullsized":"2024-09-18T07:17:15.364342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:36:44.311383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080896,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":18,"num_read_kb":13,"num_write":5351,"num_write_kb":144518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138107+0000","last_change":"2024-09-18T07:14:15.255301+0000","last_active":"2024-09-18T07:17:03.138107+0000","last_peered":"2024-09-18T07:17:03.138107+0000","last_clean":"2024-09-18T07:17:03.138107+0000","last_became_active":"2024-09-18T07:14:13.243793+0000","last_became_peered":"2024-09-18T07:14:13.243793+0000","last_unstale":"2024-09-18T07:17:03.138107+0000","last_undegraded":"2024-09-18T07:17:03.138107+0000","last_fullsized":"2024-09-18T07:17:03.138107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:24:14.077910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039863000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":121,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.147737+0000","last_change":"2024-09-18T07:14:06.881800+0000","last_active":"2024-09-18T07:17:03.147737+0000","last_peered":"2024-09-18T07:17:03.147737+0000","last_clean":"2024-09-18T07:17:03.147737+0000","last_became_active":"2024-09-18T07:14:06.880771+0000","last_became_peered":"2024-09-18T07:14:06.880771+0000","last_unstale":"2024-09-18T07:17:03.147737+0000","last_undegraded":"2024-09-18T07:17:03.147737+0000","last_fullsized":"2024-09-18T07:17:03.147737+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:05.472923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:05.472923+0000","last_clean_scrub_stamp":"2024-09-18T07:14:05.472923+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:07.064297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"25'5233","reported_seq":5249,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.374017+0000","last_change":"2024-09-18T07:14:22.280027+0000","last_active":"2024-09-18T07:17:03.374017+0000","last_peered":"2024-09-18T07:17:03.374017+0000","last_clean":"2024-09-18T07:17:03.374017+0000","last_became_active":"2024-09-18T07:14:22.279712+0000","last_became_peered":"2024-09-18T07:14:22.279712+0000","last_unstale":"2024-09-18T07:17:03.374017+0000","last_undegraded":"2024-09-18T07:17:03.374017+0000","last_fullsized":"2024-09-18T07:17:03.374017+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:35:31.397370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136970240,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5162,"num_write_kb":248325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"25'8264","reported_seq":8400,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:19.307021+0000","last_change":"2024-09-18T07:14:22.475260+0000","last_active":"2024-09-18T07:17:19.307021+0000","last_peered":"2024-09-18T07:17:19.307021+0000","last_clean":"2024-09-18T07:17:19.307021+0000","last_became_active":"2024-09-18T07:14:22.475044+0000","last_became_peered":"2024-09-18T07:14:22.475044+0000","last_unstale":"2024-09-18T07:17:19.307021+0000","last_undegraded":"2024-09-18T07:17:19.307021+0000","last_fullsized":"2024-09-18T07:17:19.307021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":8264,"log_dups_size":0,"ondisk_log_size":8264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:52:46.307809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":384,"num_read_kb":402,"num_write":5199,"num_write_kb":145227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"25'6938","reported_seq":6956,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:20.283862+0000","last_change":"2024-09-18T07:14:22.286302+0000","last_active":"2024-09-18T07:17:20.283862+0000","last_peered":"2024-09-18T07:17:20.283862+0000","last_clean":"2024-09-18T07:17:20.283862+0000","last_became_active":"2024-09-18T07:14:22.285050+0000","last_became_peered":"2024-09-18T07:14:22.285050+0000","last_unstale":"2024-09-18T07:17:20.283862+0000","last_undegraded":"2024-09-18T07:17:20.283862+0000","last_fullsized":"2024-09-18T07:17:20.283862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:36:34.664394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146120704,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":6915,"num_write_kb":222193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"24'4813","reported_seq":4845,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.168379+0000","last_change":"2024-09-18T07:14:22.284686+0000","last_active":"2024-09-18T07:17:03.168379+0000","last_peered":"2024-09-18T07:17:03.168379+0000","last_clean":"2024-09-18T07:17:03.168379+0000","last_became_active":"2024-09-18T07:14:22.284397+0000","last_became_peered":"2024-09-18T07:14:22.284397+0000","last_unstale":"2024-09-18T07:17:03.168379+0000","last_undegraded":"2024-09-18T07:17:03.168379+0000","last_fullsized":"2024-09-18T07:17:03.168379+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4813,"log_dups_size":0,"ondisk_log_size":4813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:02:38.728384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104988672,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":64,"num_read_kb":50,"num_write":4839,"num_write_kb":163526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"25'4924","reported_seq":4951,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.373267+0000","last_change":"2024-09-18T07:14:22.288110+0000","last_active":"2024-09-18T07:17:03.373267+0000","last_peered":"2024-09-18T07:17:03.373267+0000","last_clean":"2024-09-18T07:17:03.373267+0000","last_became_active":"2024-09-18T07:14:22.287513+0000","last_became_peered":"2024-09-18T07:14:22.287513+0000","last_unstale":"2024-09-18T07:17:03.373267+0000","last_undegraded":"2024-09-18T07:17:03.373267+0000","last_fullsized":"2024-09-18T07:17:03.373267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4924,"log_dups_size":0,"ondisk_log_size":4924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:02:43.799881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794368,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":43,"num_read_kb":96,"num_write":4998,"num_write_kb":181316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"25'3047","reported_seq":3065,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.318954+0000","last_change":"2024-09-18T07:14:22.289036+0000","last_active":"2024-09-18T07:17:03.318954+0000","last_peered":"2024-09-18T07:17:03.318954+0000","last_clean":"2024-09-18T07:17:03.318954+0000","last_became_active":"2024-09-18T07:14:22.288690+0000","last_became_peered":"2024-09-18T07:14:22.288690+0000","last_unstale":"2024-09-18T07:17:03.318954+0000","last_undegraded":"2024-09-18T07:17:03.318954+0000","last_fullsized":"2024-09-18T07:17:03.318954+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3047,"log_dups_size":0,"ondisk_log_size":3047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:02:01.547710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92275200,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":3030,"num_write_kb":128003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"25'3892","reported_seq":3976,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.404529+0000","last_change":"2024-09-18T07:14:22.289068+0000","last_active":"2024-09-18T07:17:03.404529+0000","last_peered":"2024-09-18T07:17:03.404529+0000","last_clean":"2024-09-18T07:17:03.404529+0000","last_became_active":"2024-09-18T07:14:22.288767+0000","last_became_peered":"2024-09-18T07:14:22.288767+0000","last_unstale":"2024-09-18T07:17:03.404529+0000","last_undegraded":"2024-09-18T07:17:03.404529+0000","last_fullsized":"2024-09-18T07:17:03.404529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:08:25.598857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106375352,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":259,"num_read_kb":194,"num_write":4137,"num_write_kb":154845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"25'5118","reported_seq":5134,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:18:08.277515+0000","last_change":"2024-09-18T07:14:22.289787+0000","last_active":"2024-09-18T07:18:08.277515+0000","last_peered":"2024-09-18T07:18:08.277515+0000","last_clean":"2024-09-18T07:18:08.277515+0000","last_became_active":"2024-09-18T07:14:22.289527+0000","last_became_peered":"2024-09-18T07:14:22.289527+0000","last_unstale":"2024-09-18T07:18:08.277515+0000","last_undegraded":"2024-09-18T07:18:08.277515+0000","last_fullsized":"2024-09-18T07:18:08.277515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5118,"log_dups_size":0,"ondisk_log_size":5118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:39:43.412358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130834432,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5068,"num_write_kb":190489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"25'3027","reported_seq":3348,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.372468+0000","last_change":"2024-09-18T07:14:22.285350+0000","last_active":"2024-09-18T07:17:03.372468+0000","last_peered":"2024-09-18T07:17:03.372468+0000","last_clean":"2024-09-18T07:17:03.372468+0000","last_became_active":"2024-09-18T07:14:22.284989+0000","last_became_peered":"2024-09-18T07:14:22.284989+0000","last_unstale":"2024-09-18T07:17:03.372468+0000","last_undegraded":"2024-09-18T07:17:03.372468+0000","last_fullsized":"2024-09-18T07:17:03.372468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3027,"log_dups_size":0,"ondisk_log_size":3027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:58:31.394769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71434240,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1022,"num_read_kb":916,"num_write":3192,"num_write_kb":144514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"24'3065","reported_seq":3104,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.147623+0000","last_change":"2024-09-18T07:14:22.290602+0000","last_active":"2024-09-18T07:17:03.147623+0000","last_peered":"2024-09-18T07:17:03.147623+0000","last_clean":"2024-09-18T07:17:03.147623+0000","last_became_active":"2024-09-18T07:14:22.290299+0000","last_became_peered":"2024-09-18T07:14:22.290299+0000","last_unstale":"2024-09-18T07:17:03.147623+0000","last_undegraded":"2024-09-18T07:17:03.147623+0000","last_fullsized":"2024-09-18T07:17:03.147623+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3065,"log_dups_size":0,"ondisk_log_size":3065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:31:47.964089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":3095,"num_write_kb":160450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"25'3705","reported_seq":3721,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.374545+0000","last_change":"2024-09-18T07:14:22.290568+0000","last_active":"2024-09-18T07:17:03.374545+0000","last_peered":"2024-09-18T07:17:03.374545+0000","last_clean":"2024-09-18T07:17:03.374545+0000","last_became_active":"2024-09-18T07:14:22.290308+0000","last_became_peered":"2024-09-18T07:14:22.290308+0000","last_unstale":"2024-09-18T07:17:03.374545+0000","last_undegraded":"2024-09-18T07:17:03.374545+0000","last_fullsized":"2024-09-18T07:17:03.374545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3705,"log_dups_size":0,"ondisk_log_size":3705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:17.589485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120979456,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3652,"num_write_kb":176387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"25'7744","reported_seq":7771,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:13.571757+0000","last_change":"2024-09-18T07:14:22.288236+0000","last_active":"2024-09-18T07:17:13.571757+0000","last_peered":"2024-09-18T07:17:13.571757+0000","last_clean":"2024-09-18T07:17:13.571757+0000","last_became_active":"2024-09-18T07:14:22.287927+0000","last_became_peered":"2024-09-18T07:14:22.287927+0000","last_unstale":"2024-09-18T07:17:13.571757+0000","last_undegraded":"2024-09-18T07:17:13.571757+0000","last_fullsized":"2024-09-18T07:17:13.571757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7744,"log_dups_size":0,"ondisk_log_size":7744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:26:51.355433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":31,"num_read_kb":87,"num_write":7741,"num_write_kb":165450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3581707016,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":2509,"num_read_kb":2520,"num_write":165325,"num_write_kb":5659033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6865756160,"data_stored":6861482136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":168164,"ondisk_log_size":168164,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1309716,"kb_used_data":1043972,"kb_used_omap":1,"kb_used_meta":265726,"kb_avail":93062124,"statfs":{"total":96636764160,"available":95295614976,"internally_reserved":0,"allocated":1069027328,"data_stored":1068306725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":272103882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408832,"kb_used_data":311024,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93963008,"statfs":{"total":96636764160,"available":96218120192,"internally_reserved":0,"allocated":318488576,"data_stored":318129409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":100137414},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542196,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1674096,"kb_used_data":1300128,"kb_used_omap":1,"kb_used_meta":373950,"kb_avail":92697744,"statfs":{"total":96636764160,"available":94922489856,"internally_reserved":0,"allocated":1331331072,"data_stored":1330396577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":382925253},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542196,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1384272,"kb_used_data":1082432,"kb_used_omap":1,"kb_used_meta":301822,"kb_avail":92987568,"statfs":{"total":96636764160,"available":95219269632,"internally_reserved":0,"allocated":1108410368,"data_stored":1107550749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":309066183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542196,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436740,"kb_used_data":338932,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93935100,"statfs":{"total":96636764160,"available":96189542400,"internally_reserved":0,"allocated":347066368,"data_stored":346699873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":100137413},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542196,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902300,"kb_used_data":676812,"kb_used_omap":1,"kb_used_meta":225470,"kb_avail":93469540,"statfs":{"total":96636764160,"available":95712808960,"internally_reserved":0,"allocated":693055488,"data_stored":692411364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":230881733},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542196,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1276088,"kb_used_data":1002024,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":93095752,"statfs":{"total":96636764160,"available":95330050048,"internally_reserved":0,"allocated":1026072576,"data_stored":1025248181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":280623557},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1182036,"kb_used_data":952388,"kb_used_omap":1,"kb_used_meta":229630,"kb_avail":93189804,"statfs":{"total":96636764160,"available":95426359296,"internally_reserved":0,"allocated":975245312,"data_stored":974453512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235141573},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1025306624,"data_stored":1024637208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":692817920,"data_stored":692329300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346832896,"data_stored":346617828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1108176896,"data_stored":1107468704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1331097600,"data_stored":1330314532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":975007744,"data_stored":974371448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":318255104,"data_stored":318047364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1068261376,"data_stored":1067695752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T07:18:20.424 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T07:18:20.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T07:18:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.521+0000 7f0389d3d640 1 Processor -- start 2024-09-18T07:18:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.521+0000 7f0389d3d640 1 -- start start 2024-09-18T07:18:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f0389d3d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0384070a80 con 0x7f0384140160 2024-09-18T07:18:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f0389d3d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03840741e0 con 0x7f03840706a0 2024-09-18T07:18:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f0389d3d640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0384073a70 con 0x7f03840730d0 2024-09-18T07:18:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f0382ffd640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f03840730d0 0x7f0384143d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:36430/0 (socket says 172.21.15.70:36430) 2024-09-18T07:18:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f0382ffd640 1 -- 172.21.15.70:0/1409738043 learned_addr learned my addr 172.21.15.70:0/1409738043 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f0383fff640 1 --1- 172.21.15.70:0/1409738043 >> v1:172.21.15.70:6789/0 conn(0x7f0384140160 0x7f0384144450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34164/0 (socket says 172.21.15.70:34164) 2024-09-18T07:18:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f03827fc640 1 -- 172.21.15.70:0/1409738043 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4270913098 0 0) 0x7f0384073a70 con 0x7f03840730d0 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f03827fc640 1 -- 172.21.15.70:0/1409738043 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f036c003520 con 0x7f03840730d0 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f03827fc640 1 -- 172.21.15.70:0/1409738043 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2761580583 0 0) 0x7f0384070a80 con 0x7f0384140160 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f03827fc640 1 -- 172.21.15.70:0/1409738043 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0384073a70 con 0x7f0384140160 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f03827fc640 1 -- 172.21.15.70:0/1409738043 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2842762449 0 0) 0x7f036c003520 con 0x7f03840730d0 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f03827fc640 1 -- 172.21.15.70:0/1409738043 >> v1:172.21.15.172:6789/0 conn(0x7f03840706a0 legacy=0x7f038413fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f03827fc640 1 -- 172.21.15.70:0/1409738043 >> v1:172.21.15.70:6789/0 conn(0x7f0384140160 legacy=0x7f0384144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f03827fc640 1 -- 172.21.15.70:0/1409738043 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03840aa550 con 0x7f03840730d0 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f03827fc640 1 -- 172.21.15.70:0/1409738043 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f0378001690 con 0x7f03840730d0 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f03827fc640 1 -- 172.21.15.70:0/1409738043 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0378001830 con 0x7f03840730d0 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.522+0000 7f03827fc640 1 -- 172.21.15.70:0/1409738043 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f0378001cc0 con 0x7f03840730d0 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.523+0000 7f0389d3d640 1 -- 172.21.15.70:0/1409738043 >> v1:172.21.15.70:6790/0 conn(0x7f03840730d0 legacy=0x7f0384143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.523+0000 7f0389d3d640 1 -- 172.21.15.70:0/1409738043 shutdown_connections 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.523+0000 7f0389d3d640 1 -- 172.21.15.70:0/1409738043 >> 172.21.15.70:0/1409738043 conn(0x7f03840b5790 msgr2=0x7f03840b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.523+0000 7f0389d3d640 1 -- 172.21.15.70:0/1409738043 shutdown_connections 2024-09-18T07:18:20.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.523+0000 7f0389d3d640 1 -- 172.21.15.70:0/1409738043 wait complete. 2024-09-18T07:18:20.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.524+0000 7f0389d3d640 1 Processor -- start 2024-09-18T07:18:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.524+0000 7f0389d3d640 1 -- start start 2024-09-18T07:18:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f0389d3d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0384139d30 con 0x7f0384140160 2024-09-18T07:18:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f0389d3d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f038413a4f0 con 0x7f03840706a0 2024-09-18T07:18:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f0389d3d640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f038413b830 con 0x7f03840730d0 2024-09-18T07:18:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f0382ffd640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f03840730d0 0x7f03841d7840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:36438/0 (socket says 172.21.15.70:36438) 2024-09-18T07:18:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f0382ffd640 1 -- 172.21.15.70:0/3035648529 learned_addr learned my addr 172.21.15.70:0/3035648529 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2500218154 0 0) 0x7f0384139d30 con 0x7f0384140160 2024-09-18T07:18:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0370003520 con 0x7f0384140160 2024-09-18T07:18:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 730879839 0 0) 0x7f038413b830 con 0x7f03840730d0 2024-09-18T07:18:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0384139d30 con 0x7f03840730d0 2024-09-18T07:18:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1037492120 0 0) 0x7f0370003520 con 0x7f0384140160 2024-09-18T07:18:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f038413b830 con 0x7f0384140160 2024-09-18T07:18:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f037c001850 con 0x7f0384140160 2024-09-18T07:18:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4239338957 0 0) 0x7f038413a4f0 con 0x7f03840706a0 2024-09-18T07:18:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0370003520 con 0x7f03840706a0 2024-09-18T07:18:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 889590827 0 0) 0x7f0384139d30 con 0x7f03840730d0 2024-09-18T07:18:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f038413a4f0 con 0x7f03840730d0 2024-09-18T07:18:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f0378001b80 con 0x7f03840730d0 2024-09-18T07:18:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2743287946 0 0) 0x7f038413b830 con 0x7f0384140160 2024-09-18T07:18:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 >> v1:172.21.15.70:6790/0 conn(0x7f03840730d0 legacy=0x7f03841d7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 >> v1:172.21.15.172:6789/0 conn(0x7f03840706a0 legacy=0x7f03841309f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0384139fd0 con 0x7f0384140160 2024-09-18T07:18:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f0389d3d640 1 -- 172.21.15.70:0/3035648529 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0384142d40 con 0x7f0384140160 2024-09-18T07:18:20.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.525+0000 7f0389d3d640 1 -- 172.21.15.70:0/3035648529 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f038413f4e0 con 0x7f0384140160 2024-09-18T07:18:20.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.526+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f037c0047e0 con 0x7f0384140160 2024-09-18T07:18:20.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.526+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f037c004b30 con 0x7f0384140160 2024-09-18T07:18:20.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.526+0000 7f0389d3d640 1 -- 172.21.15.70:0/3035648529 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0364005080 con 0x7f0384140160 2024-09-18T07:18:20.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.527+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f037c01d900 con 0x7f0384140160 2024-09-18T07:18:20.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.528+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f037c096100 con 0x7f0384140160 2024-09-18T07:18:20.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.532+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f037c05e960 con 0x7f0384140160 2024-09-18T07:18:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.706+0000 7f0389d3d640 1 -- 172.21.15.70:0/3035648529 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0364000b90 con 0x7f0370079880 2024-09-18T07:18:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.708+0000 7f036bfff640 1 -- 172.21.15.70:0/3035648529 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117442 (unknown 2965378022 0 272469850) 0x7f0364000b90 con 0x7f0370079880 2024-09-18T07:18:20.711 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:18:20.712 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-18T07:18:20.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.710+0000 7f0389d3d640 1 -- 172.21.15.70:0/3035648529 >> v1:172.21.15.70:6816/1657990791 conn(0x7f0370079880 legacy=0x7f037007bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.710+0000 7f0389d3d640 1 -- 172.21.15.70:0/3035648529 >> v1:172.21.15.70:6789/0 conn(0x7f0384140160 legacy=0x7f03841dfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.710+0000 7f0389d3d640 1 -- 172.21.15.70:0/3035648529 shutdown_connections 2024-09-18T07:18:20.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.710+0000 7f0389d3d640 1 -- 172.21.15.70:0/3035648529 >> 172.21.15.70:0/3035648529 conn(0x7f03840b5790 msgr2=0x7f0384140ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:20.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.711+0000 7f0389d3d640 1 -- 172.21.15.70:0/3035648529 shutdown_connections 2024-09-18T07:18:20.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.711+0000 7f0389d3d640 1 -- 172.21.15.70:0/3035648529 wait complete. 2024-09-18T07:18:20.724 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-18T07:18:19.496357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3582297403,"num_objects":886,"num_object_clones":0,"num_object_copies":1772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":886,"num_whiteouts":0,"num_read":2575,"num_read_kb":2575,"num_write":165428,"num_write_kb":5660663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":168239,"ondisk_log_size":168239,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8574080,"kb_used_data":6707712,"kb_used_omap":12,"kb_used_meta":1866227,"kb_avail":746400640,"statfs":{"total":773094113280,"available":764314255360,"internally_reserved":0,"allocated":6868697088,"data_stored":6863196390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1911017008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":69632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002343"},"pg_stats":[{"pgid":"3.1f","version":"24'3617","reported_seq":3635,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.171919+0000","last_change":"2024-09-18T07:14:22.286661+0000","last_active":"2024-09-18T07:17:03.171919+0000","last_peered":"2024-09-18T07:17:03.171919+0000","last_clean":"2024-09-18T07:17:03.171919+0000","last_became_active":"2024-09-18T07:14:22.286401+0000","last_became_peered":"2024-09-18T07:14:22.286401+0000","last_unstale":"2024-09-18T07:17:03.171919+0000","last_undegraded":"2024-09-18T07:17:03.171919+0000","last_fullsized":"2024-09-18T07:17:03.171919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3617,"log_dups_size":0,"ondisk_log_size":3617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:26:38.664961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95160672,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":3623,"num_write_kb":151525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"25'6414","reported_seq":6531,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:09.318453+0000","last_change":"2024-09-18T07:14:22.284740+0000","last_active":"2024-09-18T07:17:09.318453+0000","last_peered":"2024-09-18T07:17:09.318453+0000","last_clean":"2024-09-18T07:17:09.318453+0000","last_became_active":"2024-09-18T07:14:22.284487+0000","last_became_peered":"2024-09-18T07:14:22.284487+0000","last_unstale":"2024-09-18T07:17:09.318453+0000","last_undegraded":"2024-09-18T07:17:09.318453+0000","last_fullsized":"2024-09-18T07:17:09.318453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":6414,"log_dups_size":0,"ondisk_log_size":6414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:23:39.072202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377808,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":6397,"num_write_kb":190429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"25'7587","reported_seq":7636,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:14.357682+0000","last_change":"2024-09-18T07:14:22.288902+0000","last_active":"2024-09-18T07:17:14.357682+0000","last_peered":"2024-09-18T07:17:14.357682+0000","last_clean":"2024-09-18T07:17:14.357682+0000","last_became_active":"2024-09-18T07:14:22.288441+0000","last_became_peered":"2024-09-18T07:14:22.288441+0000","last_unstale":"2024-09-18T07:17:14.357682+0000","last_undegraded":"2024-09-18T07:17:14.357682+0000","last_fullsized":"2024-09-18T07:17:14.357682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7587,"log_dups_size":0,"ondisk_log_size":7587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:04:30.016419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167903232,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":127,"num_read_kb":103,"num_write":7620,"num_write_kb":243849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"24'3255","reported_seq":3290,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138103+0000","last_change":"2024-09-18T07:14:22.288842+0000","last_active":"2024-09-18T07:17:03.138103+0000","last_peered":"2024-09-18T07:17:03.138103+0000","last_clean":"2024-09-18T07:17:03.138103+0000","last_became_active":"2024-09-18T07:14:22.287853+0000","last_became_peered":"2024-09-18T07:14:22.287853+0000","last_unstale":"2024-09-18T07:17:03.138103+0000","last_undegraded":"2024-09-18T07:17:03.138103+0000","last_fullsized":"2024-09-18T07:17:03.138103+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3255,"log_dups_size":0,"ondisk_log_size":3255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:14.436120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88211456,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":79,"num_read_kb":61,"num_write":3296,"num_write_kb":156739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"25'3829","reported_seq":3848,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:18:08.280825+0000","last_change":"2024-09-18T07:14:22.286284+0000","last_active":"2024-09-18T07:18:08.280825+0000","last_peered":"2024-09-18T07:18:08.280825+0000","last_clean":"2024-09-18T07:18:08.280825+0000","last_became_active":"2024-09-18T07:14:22.286018+0000","last_became_peered":"2024-09-18T07:14:22.286018+0000","last_unstale":"2024-09-18T07:18:08.280825+0000","last_undegraded":"2024-09-18T07:18:08.280825+0000","last_fullsized":"2024-09-18T07:18:08.280825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3829,"log_dups_size":0,"ondisk_log_size":3829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:03:53.111506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91650400,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":3788,"num_write_kb":184441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"25'5565","reported_seq":5585,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:07.526513+0000","last_change":"2024-09-18T07:14:22.280275+0000","last_active":"2024-09-18T07:17:07.526513+0000","last_peered":"2024-09-18T07:17:07.526513+0000","last_clean":"2024-09-18T07:17:07.526513+0000","last_became_active":"2024-09-18T07:14:22.279826+0000","last_became_peered":"2024-09-18T07:14:22.279826+0000","last_unstale":"2024-09-18T07:17:07.526513+0000","last_undegraded":"2024-09-18T07:17:07.526513+0000","last_fullsized":"2024-09-18T07:17:07.526513+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5565,"log_dups_size":0,"ondisk_log_size":5565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:19:40.864770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":15,"num_read_kb":75,"num_write":5550,"num_write_kb":155179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"25'4743","reported_seq":4775,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:18.223911+0000","last_change":"2024-09-18T07:14:22.280164+0000","last_active":"2024-09-18T07:17:18.223911+0000","last_peered":"2024-09-18T07:17:18.223911+0000","last_clean":"2024-09-18T07:17:18.223911+0000","last_became_active":"2024-09-18T07:14:22.279885+0000","last_became_peered":"2024-09-18T07:14:22.279885+0000","last_unstale":"2024-09-18T07:17:18.223911+0000","last_undegraded":"2024-09-18T07:17:18.223911+0000","last_fullsized":"2024-09-18T07:17:18.223911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:08:22.395662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":4713,"num_write_kb":147181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"25'8007","reported_seq":8024,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:18.366347+0000","last_change":"2024-09-18T07:14:22.283501+0000","last_active":"2024-09-18T07:17:18.366347+0000","last_peered":"2024-09-18T07:17:18.366347+0000","last_clean":"2024-09-18T07:17:18.366347+0000","last_became_active":"2024-09-18T07:14:22.283231+0000","last_became_peered":"2024-09-18T07:14:22.283231+0000","last_unstale":"2024-09-18T07:17:18.366347+0000","last_undegraded":"2024-09-18T07:17:18.366347+0000","last_fullsized":"2024-09-18T07:17:18.366347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":8007,"log_dups_size":0,"ondisk_log_size":8007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:49:47.239612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125895176,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":8013,"num_write_kb":214525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"25'7235","reported_seq":7255,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:21.527844+0000","last_change":"2024-09-18T07:14:22.286153+0000","last_active":"2024-09-18T07:17:21.527844+0000","last_peered":"2024-09-18T07:17:21.527844+0000","last_clean":"2024-09-18T07:17:21.527844+0000","last_became_active":"2024-09-18T07:14:22.285884+0000","last_became_peered":"2024-09-18T07:14:22.285884+0000","last_unstale":"2024-09-18T07:17:21.527844+0000","last_undegraded":"2024-09-18T07:17:21.527844+0000","last_fullsized":"2024-09-18T07:17:21.527844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7235,"log_dups_size":0,"ondisk_log_size":7235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:36:41.719570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135659520,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":7238,"num_write_kb":200827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"25'3408","reported_seq":3432,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.319245+0000","last_change":"2024-09-18T07:14:22.289679+0000","last_active":"2024-09-18T07:17:03.319245+0000","last_peered":"2024-09-18T07:17:03.319245+0000","last_clean":"2024-09-18T07:17:03.319245+0000","last_became_active":"2024-09-18T07:14:22.289372+0000","last_became_peered":"2024-09-18T07:14:22.289372+0000","last_unstale":"2024-09-18T07:17:03.319245+0000","last_undegraded":"2024-09-18T07:17:03.319245+0000","last_fullsized":"2024-09-18T07:17:03.319245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3408,"log_dups_size":0,"ondisk_log_size":3408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:59:31.222383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112003944,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":22,"num_read_kb":143,"num_write":3412,"num_write_kb":151531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"25'8025","reported_seq":8044,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:17.126686+0000","last_change":"2024-09-18T07:14:22.288942+0000","last_active":"2024-09-18T07:17:17.126686+0000","last_peered":"2024-09-18T07:17:17.126686+0000","last_clean":"2024-09-18T07:17:17.126686+0000","last_became_active":"2024-09-18T07:14:22.288634+0000","last_became_peered":"2024-09-18T07:14:22.288634+0000","last_unstale":"2024-09-18T07:17:17.126686+0000","last_undegraded":"2024-09-18T07:17:17.126686+0000","last_fullsized":"2024-09-18T07:17:17.126686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":8025,"log_dups_size":0,"ondisk_log_size":8025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:34:15.636211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8027,"num_write_kb":201731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"25'4872","reported_seq":4889,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:10.976186+0000","last_change":"2024-09-18T07:14:22.291854+0000","last_active":"2024-09-18T07:17:10.976186+0000","last_peered":"2024-09-18T07:17:10.976186+0000","last_clean":"2024-09-18T07:17:10.976186+0000","last_became_active":"2024-09-18T07:14:22.291551+0000","last_became_peered":"2024-09-18T07:14:22.291551+0000","last_unstale":"2024-09-18T07:17:10.976186+0000","last_undegraded":"2024-09-18T07:17:10.976186+0000","last_fullsized":"2024-09-18T07:17:10.976186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4872,"log_dups_size":0,"ondisk_log_size":4872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:54:13.729656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303680,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4826,"num_write_kb":160133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.171591+0000","last_change":"2024-09-18T07:14:15.808642+0000","last_active":"2024-09-18T07:17:03.171591+0000","last_peered":"2024-09-18T07:17:03.171591+0000","last_clean":"2024-09-18T07:17:03.171591+0000","last_became_active":"2024-09-18T07:14:13.251609+0000","last_became_peered":"2024-09-18T07:14:13.251609+0000","last_unstale":"2024-09-18T07:17:03.171591+0000","last_undegraded":"2024-09-18T07:17:03.171591+0000","last_fullsized":"2024-09-18T07:17:03.171591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:01.748551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040903899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"25'7878","reported_seq":7894,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:18:08.279680+0000","last_change":"2024-09-18T07:14:22.283241+0000","last_active":"2024-09-18T07:18:08.279680+0000","last_peered":"2024-09-18T07:18:08.279680+0000","last_clean":"2024-09-18T07:18:08.279680+0000","last_became_active":"2024-09-18T07:14:22.282926+0000","last_became_peered":"2024-09-18T07:14:22.282926+0000","last_unstale":"2024-09-18T07:18:08.279680+0000","last_undegraded":"2024-09-18T07:18:08.279680+0000","last_fullsized":"2024-09-18T07:18:08.279680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7878,"log_dups_size":0,"ondisk_log_size":7878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:11:58.849404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141926400,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7854,"num_write_kb":222187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138640+0000","last_change":"2024-09-18T07:14:15.249860+0000","last_active":"2024-09-18T07:17:03.138640+0000","last_peered":"2024-09-18T07:17:03.138640+0000","last_clean":"2024-09-18T07:17:03.138640+0000","last_became_active":"2024-09-18T07:14:13.251585+0000","last_became_peered":"2024-09-18T07:14:13.251585+0000","last_unstale":"2024-09-18T07:17:03.138640+0000","last_undegraded":"2024-09-18T07:17:03.138640+0000","last_fullsized":"2024-09-18T07:17:03.138640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:02:29.914389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039885799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"25'4220","reported_seq":4246,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.375214+0000","last_change":"2024-09-18T07:14:22.288077+0000","last_active":"2024-09-18T07:17:03.375214+0000","last_peered":"2024-09-18T07:17:03.375214+0000","last_clean":"2024-09-18T07:17:03.375214+0000","last_became_active":"2024-09-18T07:14:22.287843+0000","last_became_peered":"2024-09-18T07:14:22.287843+0000","last_unstale":"2024-09-18T07:17:03.375214+0000","last_undegraded":"2024-09-18T07:17:03.375214+0000","last_fullsized":"2024-09-18T07:17:03.375214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4220,"log_dups_size":0,"ondisk_log_size":4220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:13.075300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141164544,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":4202,"num_write_kb":175065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.148047+0000","last_change":"2024-09-18T07:14:15.603034+0000","last_active":"2024-09-18T07:17:03.148047+0000","last_peered":"2024-09-18T07:17:03.148047+0000","last_clean":"2024-09-18T07:17:03.148047+0000","last_became_active":"2024-09-18T07:14:13.957168+0000","last_became_peered":"2024-09-18T07:14:13.957168+0000","last_unstale":"2024-09-18T07:17:03.148047+0000","last_undegraded":"2024-09-18T07:17:03.148047+0000","last_fullsized":"2024-09-18T07:17:03.148047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:57:48.846502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048630800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"25'7512","reported_seq":7528,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:20.431132+0000","last_change":"2024-09-18T07:14:22.475409+0000","last_active":"2024-09-18T07:17:20.431132+0000","last_peered":"2024-09-18T07:17:20.431132+0000","last_clean":"2024-09-18T07:17:20.431132+0000","last_became_active":"2024-09-18T07:14:22.475086+0000","last_became_peered":"2024-09-18T07:14:22.475086+0000","last_unstale":"2024-09-18T07:17:20.431132+0000","last_undegraded":"2024-09-18T07:17:20.431132+0000","last_fullsized":"2024-09-18T07:17:20.431132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7512,"log_dups_size":0,"ondisk_log_size":7512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:33:15.601379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7506,"num_write_kb":189960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138588+0000","last_change":"2024-09-18T07:14:15.249863+0000","last_active":"2024-09-18T07:17:03.138588+0000","last_peered":"2024-09-18T07:17:03.138588+0000","last_clean":"2024-09-18T07:17:03.138588+0000","last_became_active":"2024-09-18T07:14:13.956929+0000","last_became_peered":"2024-09-18T07:14:13.956929+0000","last_unstale":"2024-09-18T07:17:03.138588+0000","last_undegraded":"2024-09-18T07:17:03.138588+0000","last_fullsized":"2024-09-18T07:17:03.138588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:11:55.607274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045339399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'5270","reported_seq":5286,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:22.633206+0000","last_change":"2024-09-18T07:14:22.288367+0000","last_active":"2024-09-18T07:17:22.633206+0000","last_peered":"2024-09-18T07:17:22.633206+0000","last_clean":"2024-09-18T07:17:22.633206+0000","last_became_active":"2024-09-18T07:14:22.288096+0000","last_became_peered":"2024-09-18T07:14:22.288096+0000","last_unstale":"2024-09-18T07:17:22.633206+0000","last_undegraded":"2024-09-18T07:17:22.633206+0000","last_fullsized":"2024-09-18T07:17:22.633206+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5270,"log_dups_size":0,"ondisk_log_size":5270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:39:40.638354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5267,"num_write_kb":201861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'3608","reported_seq":3634,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.140959+0000","last_change":"2024-09-18T07:14:22.284180+0000","last_active":"2024-09-18T07:17:03.140959+0000","last_peered":"2024-09-18T07:17:03.140959+0000","last_clean":"2024-09-18T07:17:03.140959+0000","last_became_active":"2024-09-18T07:14:22.283205+0000","last_became_peered":"2024-09-18T07:14:22.283205+0000","last_unstale":"2024-09-18T07:17:03.140959+0000","last_undegraded":"2024-09-18T07:17:03.140959+0000","last_fullsized":"2024-09-18T07:17:03.140959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3608,"log_dups_size":0,"ondisk_log_size":3608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:36:44.407465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92283408,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":37,"num_read_kb":29,"num_write":3631,"num_write_kb":171797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138107+0000","last_change":"2024-09-18T07:14:15.255874+0000","last_active":"2024-09-18T07:17:03.138107+0000","last_peered":"2024-09-18T07:17:03.138107+0000","last_clean":"2024-09-18T07:17:03.138107+0000","last_became_active":"2024-09-18T07:14:13.243579+0000","last_became_peered":"2024-09-18T07:14:13.243579+0000","last_unstale":"2024-09-18T07:17:03.138107+0000","last_undegraded":"2024-09-18T07:17:03.138107+0000","last_fullsized":"2024-09-18T07:17:03.138107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:28.722433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096575000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"25'4956","reported_seq":4980,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.371555+0000","last_change":"2024-09-18T07:14:22.475468+0000","last_active":"2024-09-18T07:17:03.371555+0000","last_peered":"2024-09-18T07:17:03.371555+0000","last_clean":"2024-09-18T07:17:03.371555+0000","last_became_active":"2024-09-18T07:14:22.474980+0000","last_became_peered":"2024-09-18T07:14:22.474980+0000","last_unstale":"2024-09-18T07:17:03.371555+0000","last_undegraded":"2024-09-18T07:17:03.371555+0000","last_fullsized":"2024-09-18T07:17:03.371555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4956,"log_dups_size":0,"ondisk_log_size":4956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:38:28.516038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124452864,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4927,"num_write_kb":170572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138277+0000","last_change":"2024-09-18T07:14:15.398157+0000","last_active":"2024-09-18T07:17:03.138277+0000","last_peered":"2024-09-18T07:17:03.138277+0000","last_clean":"2024-09-18T07:17:03.138277+0000","last_became_active":"2024-09-18T07:14:13.392213+0000","last_became_peered":"2024-09-18T07:14:13.392213+0000","last_unstale":"2024-09-18T07:17:03.138277+0000","last_undegraded":"2024-09-18T07:17:03.138277+0000","last_fullsized":"2024-09-18T07:17:03.138277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:47:04.187501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000312059,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"25'3057","reported_seq":3084,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.323804+0000","last_change":"2024-09-18T07:14:22.286270+0000","last_active":"2024-09-18T07:17:03.323804+0000","last_peered":"2024-09-18T07:17:03.323804+0000","last_clean":"2024-09-18T07:17:03.323804+0000","last_became_active":"2024-09-18T07:14:22.286007+0000","last_became_peered":"2024-09-18T07:14:22.286007+0000","last_unstale":"2024-09-18T07:17:03.323804+0000","last_undegraded":"2024-09-18T07:17:03.323804+0000","last_fullsized":"2024-09-18T07:17:03.323804+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3057,"log_dups_size":0,"ondisk_log_size":3057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:21:48.710903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92405760,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":3056,"num_write_kb":144258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.147637+0000","last_change":"2024-09-18T07:14:15.603018+0000","last_active":"2024-09-18T07:17:03.147637+0000","last_peered":"2024-09-18T07:17:03.147637+0000","last_clean":"2024-09-18T07:17:03.147637+0000","last_became_active":"2024-09-18T07:14:13.251213+0000","last_became_peered":"2024-09-18T07:14:13.251213+0000","last_unstale":"2024-09-18T07:17:03.147637+0000","last_undegraded":"2024-09-18T07:17:03.147637+0000","last_fullsized":"2024-09-18T07:17:03.147637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:43:22.952115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046971900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"25'5336","reported_seq":5357,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:15.364342+0000","last_change":"2024-09-18T07:14:22.284250+0000","last_active":"2024-09-18T07:17:15.364342+0000","last_peered":"2024-09-18T07:17:15.364342+0000","last_clean":"2024-09-18T07:17:15.364342+0000","last_became_active":"2024-09-18T07:14:22.283928+0000","last_became_peered":"2024-09-18T07:14:22.283928+0000","last_unstale":"2024-09-18T07:17:15.364342+0000","last_undegraded":"2024-09-18T07:17:15.364342+0000","last_fullsized":"2024-09-18T07:17:15.364342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:36:44.311383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080896,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":18,"num_read_kb":13,"num_write":5351,"num_write_kb":144518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.138107+0000","last_change":"2024-09-18T07:14:15.255301+0000","last_active":"2024-09-18T07:17:03.138107+0000","last_peered":"2024-09-18T07:17:03.138107+0000","last_clean":"2024-09-18T07:17:03.138107+0000","last_became_active":"2024-09-18T07:14:13.243793+0000","last_became_peered":"2024-09-18T07:14:13.243793+0000","last_unstale":"2024-09-18T07:17:03.138107+0000","last_undegraded":"2024-09-18T07:17:03.138107+0000","last_fullsized":"2024-09-18T07:17:03.138107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:24:14.077910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039863000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":121,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.147737+0000","last_change":"2024-09-18T07:14:06.881800+0000","last_active":"2024-09-18T07:17:03.147737+0000","last_peered":"2024-09-18T07:17:03.147737+0000","last_clean":"2024-09-18T07:17:03.147737+0000","last_became_active":"2024-09-18T07:14:06.880771+0000","last_became_peered":"2024-09-18T07:14:06.880771+0000","last_unstale":"2024-09-18T07:17:03.147737+0000","last_undegraded":"2024-09-18T07:17:03.147737+0000","last_fullsized":"2024-09-18T07:17:03.147737+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:05.472923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:05.472923+0000","last_clean_scrub_stamp":"2024-09-18T07:14:05.472923+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:07.064297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"25'5233","reported_seq":5249,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.374017+0000","last_change":"2024-09-18T07:14:22.280027+0000","last_active":"2024-09-18T07:17:03.374017+0000","last_peered":"2024-09-18T07:17:03.374017+0000","last_clean":"2024-09-18T07:17:03.374017+0000","last_became_active":"2024-09-18T07:14:22.279712+0000","last_became_peered":"2024-09-18T07:14:22.279712+0000","last_unstale":"2024-09-18T07:17:03.374017+0000","last_undegraded":"2024-09-18T07:17:03.374017+0000","last_fullsized":"2024-09-18T07:17:03.374017+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:35:31.397370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136970240,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5162,"num_write_kb":248325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"25'8264","reported_seq":8400,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:19.307021+0000","last_change":"2024-09-18T07:14:22.475260+0000","last_active":"2024-09-18T07:17:19.307021+0000","last_peered":"2024-09-18T07:17:19.307021+0000","last_clean":"2024-09-18T07:17:19.307021+0000","last_became_active":"2024-09-18T07:14:22.475044+0000","last_became_peered":"2024-09-18T07:14:22.475044+0000","last_unstale":"2024-09-18T07:17:19.307021+0000","last_undegraded":"2024-09-18T07:17:19.307021+0000","last_fullsized":"2024-09-18T07:17:19.307021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":8264,"log_dups_size":0,"ondisk_log_size":8264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:52:46.307809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":384,"num_read_kb":402,"num_write":5199,"num_write_kb":145227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"25'6938","reported_seq":6956,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:20.283862+0000","last_change":"2024-09-18T07:14:22.286302+0000","last_active":"2024-09-18T07:17:20.283862+0000","last_peered":"2024-09-18T07:17:20.283862+0000","last_clean":"2024-09-18T07:17:20.283862+0000","last_became_active":"2024-09-18T07:14:22.285050+0000","last_became_peered":"2024-09-18T07:14:22.285050+0000","last_unstale":"2024-09-18T07:17:20.283862+0000","last_undegraded":"2024-09-18T07:17:20.283862+0000","last_fullsized":"2024-09-18T07:17:20.283862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:36:34.664394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146120704,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":6915,"num_write_kb":222193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"24'4813","reported_seq":4845,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.168379+0000","last_change":"2024-09-18T07:14:22.284686+0000","last_active":"2024-09-18T07:17:03.168379+0000","last_peered":"2024-09-18T07:17:03.168379+0000","last_clean":"2024-09-18T07:17:03.168379+0000","last_became_active":"2024-09-18T07:14:22.284397+0000","last_became_peered":"2024-09-18T07:14:22.284397+0000","last_unstale":"2024-09-18T07:17:03.168379+0000","last_undegraded":"2024-09-18T07:17:03.168379+0000","last_fullsized":"2024-09-18T07:17:03.168379+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4813,"log_dups_size":0,"ondisk_log_size":4813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:02:38.728384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104988672,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":64,"num_read_kb":50,"num_write":4839,"num_write_kb":163526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"25'4924","reported_seq":4951,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.373267+0000","last_change":"2024-09-18T07:14:22.288110+0000","last_active":"2024-09-18T07:17:03.373267+0000","last_peered":"2024-09-18T07:17:03.373267+0000","last_clean":"2024-09-18T07:17:03.373267+0000","last_became_active":"2024-09-18T07:14:22.287513+0000","last_became_peered":"2024-09-18T07:14:22.287513+0000","last_unstale":"2024-09-18T07:17:03.373267+0000","last_undegraded":"2024-09-18T07:17:03.373267+0000","last_fullsized":"2024-09-18T07:17:03.373267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4924,"log_dups_size":0,"ondisk_log_size":4924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:02:43.799881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794368,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":43,"num_read_kb":96,"num_write":4998,"num_write_kb":181316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"25'3047","reported_seq":3065,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.318954+0000","last_change":"2024-09-18T07:14:22.289036+0000","last_active":"2024-09-18T07:17:03.318954+0000","last_peered":"2024-09-18T07:17:03.318954+0000","last_clean":"2024-09-18T07:17:03.318954+0000","last_became_active":"2024-09-18T07:14:22.288690+0000","last_became_peered":"2024-09-18T07:14:22.288690+0000","last_unstale":"2024-09-18T07:17:03.318954+0000","last_undegraded":"2024-09-18T07:17:03.318954+0000","last_fullsized":"2024-09-18T07:17:03.318954+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3047,"log_dups_size":0,"ondisk_log_size":3047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:02:01.547710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92275200,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":3030,"num_write_kb":128003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"25'3892","reported_seq":3976,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.404529+0000","last_change":"2024-09-18T07:14:22.289068+0000","last_active":"2024-09-18T07:17:03.404529+0000","last_peered":"2024-09-18T07:17:03.404529+0000","last_clean":"2024-09-18T07:17:03.404529+0000","last_became_active":"2024-09-18T07:14:22.288767+0000","last_became_peered":"2024-09-18T07:14:22.288767+0000","last_unstale":"2024-09-18T07:17:03.404529+0000","last_undegraded":"2024-09-18T07:17:03.404529+0000","last_fullsized":"2024-09-18T07:17:03.404529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:08:25.598857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106375352,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":259,"num_read_kb":194,"num_write":4137,"num_write_kb":154845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"25'5118","reported_seq":5134,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:18:08.277515+0000","last_change":"2024-09-18T07:14:22.289787+0000","last_active":"2024-09-18T07:18:08.277515+0000","last_peered":"2024-09-18T07:18:08.277515+0000","last_clean":"2024-09-18T07:18:08.277515+0000","last_became_active":"2024-09-18T07:14:22.289527+0000","last_became_peered":"2024-09-18T07:14:22.289527+0000","last_unstale":"2024-09-18T07:18:08.277515+0000","last_undegraded":"2024-09-18T07:18:08.277515+0000","last_fullsized":"2024-09-18T07:18:08.277515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5118,"log_dups_size":0,"ondisk_log_size":5118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:39:43.412358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130834432,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5068,"num_write_kb":190489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"25'3027","reported_seq":3348,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.372468+0000","last_change":"2024-09-18T07:14:22.285350+0000","last_active":"2024-09-18T07:17:03.372468+0000","last_peered":"2024-09-18T07:17:03.372468+0000","last_clean":"2024-09-18T07:17:03.372468+0000","last_became_active":"2024-09-18T07:14:22.284989+0000","last_became_peered":"2024-09-18T07:14:22.284989+0000","last_unstale":"2024-09-18T07:17:03.372468+0000","last_undegraded":"2024-09-18T07:17:03.372468+0000","last_fullsized":"2024-09-18T07:17:03.372468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3027,"log_dups_size":0,"ondisk_log_size":3027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:58:31.394769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71434240,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1022,"num_read_kb":916,"num_write":3192,"num_write_kb":144514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"24'3065","reported_seq":3104,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.147623+0000","last_change":"2024-09-18T07:14:22.290602+0000","last_active":"2024-09-18T07:17:03.147623+0000","last_peered":"2024-09-18T07:17:03.147623+0000","last_clean":"2024-09-18T07:17:03.147623+0000","last_became_active":"2024-09-18T07:14:22.290299+0000","last_became_peered":"2024-09-18T07:14:22.290299+0000","last_unstale":"2024-09-18T07:17:03.147623+0000","last_undegraded":"2024-09-18T07:17:03.147623+0000","last_fullsized":"2024-09-18T07:17:03.147623+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3065,"log_dups_size":0,"ondisk_log_size":3065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:31:47.964089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":3095,"num_write_kb":160450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"25'3705","reported_seq":3721,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:03.374545+0000","last_change":"2024-09-18T07:14:22.290568+0000","last_active":"2024-09-18T07:17:03.374545+0000","last_peered":"2024-09-18T07:17:03.374545+0000","last_clean":"2024-09-18T07:17:03.374545+0000","last_became_active":"2024-09-18T07:14:22.290308+0000","last_became_peered":"2024-09-18T07:14:22.290308+0000","last_unstale":"2024-09-18T07:17:03.374545+0000","last_undegraded":"2024-09-18T07:17:03.374545+0000","last_fullsized":"2024-09-18T07:17:03.374545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3705,"log_dups_size":0,"ondisk_log_size":3705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:17.589485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120979456,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3652,"num_write_kb":176387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"25'7744","reported_seq":7771,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T07:17:13.571757+0000","last_change":"2024-09-18T07:14:22.288236+0000","last_active":"2024-09-18T07:17:13.571757+0000","last_peered":"2024-09-18T07:17:13.571757+0000","last_clean":"2024-09-18T07:17:13.571757+0000","last_became_active":"2024-09-18T07:14:22.287927+0000","last_became_peered":"2024-09-18T07:14:22.287927+0000","last_unstale":"2024-09-18T07:17:13.571757+0000","last_undegraded":"2024-09-18T07:17:13.571757+0000","last_fullsized":"2024-09-18T07:17:13.571757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7744,"log_dups_size":0,"ondisk_log_size":7744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:26:51.355433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":31,"num_read_kb":87,"num_write":7741,"num_write_kb":165450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3581707016,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":2509,"num_read_kb":2520,"num_write":165325,"num_write_kb":5659033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6865756160,"data_stored":6861482136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":168164,"ondisk_log_size":168164,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1309716,"kb_used_data":1043972,"kb_used_omap":1,"kb_used_meta":265726,"kb_avail":93062124,"statfs":{"total":96636764160,"available":95295614976,"internally_reserved":0,"allocated":1069027328,"data_stored":1068306725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":272103882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408832,"kb_used_data":311024,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93963008,"statfs":{"total":96636764160,"available":96218120192,"internally_reserved":0,"allocated":318488576,"data_stored":318129409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":100137414},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542196,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1674096,"kb_used_data":1300128,"kb_used_omap":1,"kb_used_meta":373950,"kb_avail":92697744,"statfs":{"total":96636764160,"available":94922489856,"internally_reserved":0,"allocated":1331331072,"data_stored":1330396577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":382925253},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542196,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1384272,"kb_used_data":1082432,"kb_used_omap":1,"kb_used_meta":301822,"kb_avail":92987568,"statfs":{"total":96636764160,"available":95219269632,"internally_reserved":0,"allocated":1108410368,"data_stored":1107550749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":309066183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542196,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436740,"kb_used_data":338932,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93935100,"statfs":{"total":96636764160,"available":96189542400,"internally_reserved":0,"allocated":347066368,"data_stored":346699873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":100137413},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542196,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902300,"kb_used_data":676812,"kb_used_omap":1,"kb_used_meta":225470,"kb_avail":93469540,"statfs":{"total":96636764160,"available":95712808960,"internally_reserved":0,"allocated":693055488,"data_stored":692411364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":230881733},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542196,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1276088,"kb_used_data":1002024,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":93095752,"statfs":{"total":96636764160,"available":95330050048,"internally_reserved":0,"allocated":1026072576,"data_stored":1025248181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":280623557},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1182036,"kb_used_data":952388,"kb_used_omap":1,"kb_used_meta":229630,"kb_avail":93189804,"statfs":{"total":96636764160,"available":95426359296,"internally_reserved":0,"allocated":975245312,"data_stored":974453512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235141573},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1025306624,"data_stored":1024637208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":692817920,"data_stored":692329300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346832896,"data_stored":346617828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1108176896,"data_stored":1107468704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1331097600,"data_stored":1330314532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":975007744,"data_stored":974371448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":318255104,"data_stored":318047364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1068261376,"data_stored":1067695752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T07:18:20.726 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T07:18:20.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.820+0000 7f7ef3707640 1 Processor -- start 2024-09-18T07:18:20.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.820+0000 7f7ef3707640 1 -- start start 2024-09-18T07:18:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ef3707640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eec1441f0 con 0x7f7eec13f9f0 2024-09-18T07:18:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ef3707640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eec0aae80 con 0x7f7eec072ea0 2024-09-18T07:18:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ef3707640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eec140f40 con 0x7f7eec072ac0 2024-09-18T07:18:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ef14a3640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f7eec072ac0 0x7f7eec143ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:36448/0 (socket says 172.21.15.70:36448) 2024-09-18T07:18:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ef14a3640 1 -- 172.21.15.70:0/2464256293 learned_addr learned my addr 172.21.15.70:0/2464256293 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ee3fff640 1 -- 172.21.15.70:0/2464256293 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139691215 0 0) 0x7f7eec140f40 con 0x7f7eec072ac0 2024-09-18T07:18:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ee3fff640 1 -- 172.21.15.70:0/2464256293 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ed8003520 con 0x7f7eec072ac0 2024-09-18T07:18:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ee3fff640 1 -- 172.21.15.70:0/2464256293 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 620304253 0 0) 0x7f7eec1441f0 con 0x7f7eec13f9f0 2024-09-18T07:18:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ee3fff640 1 -- 172.21.15.70:0/2464256293 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7eec140f40 con 0x7f7eec13f9f0 2024-09-18T07:18:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ee3fff640 1 -- 172.21.15.70:0/2464256293 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4082130723 0 0) 0x7f7ed8003520 con 0x7f7eec072ac0 2024-09-18T07:18:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ee3fff640 1 -- 172.21.15.70:0/2464256293 >> v1:172.21.15.172:6789/0 conn(0x7f7eec072ea0 legacy=0x7f7eec13f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ee3fff640 1 -- 172.21.15.70:0/2464256293 >> v1:172.21.15.70:6789/0 conn(0x7f7eec13f9f0 legacy=0x7f7eec14b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ee3fff640 1 -- 172.21.15.70:0/2464256293 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec073f80 con 0x7f7eec072ac0 2024-09-18T07:18:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.821+0000 7f7ee3fff640 1 -- 172.21.15.70:0/2464256293 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7ee8001690 con 0x7f7eec072ac0 2024-09-18T07:18:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.822+0000 7f7ee3fff640 1 -- 172.21.15.70:0/2464256293 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7ee80018b0 con 0x7f7eec072ac0 2024-09-18T07:18:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.822+0000 7f7ee3fff640 1 -- 172.21.15.70:0/2464256293 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7ee8001d40 con 0x7f7eec072ac0 2024-09-18T07:18:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.822+0000 7f7ef3707640 1 -- 172.21.15.70:0/2464256293 >> v1:172.21.15.70:6790/0 conn(0x7f7eec072ac0 legacy=0x7f7eec143ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.822+0000 7f7ef3707640 1 -- 172.21.15.70:0/2464256293 shutdown_connections 2024-09-18T07:18:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.822+0000 7f7ef3707640 1 -- 172.21.15.70:0/2464256293 >> 172.21.15.70:0/2464256293 conn(0x7f7eec0b5790 msgr2=0x7f7eec0b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:20.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.822+0000 7f7ef3707640 1 -- 172.21.15.70:0/2464256293 shutdown_connections 2024-09-18T07:18:20.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.822+0000 7f7ef3707640 1 -- 172.21.15.70:0/2464256293 wait complete. 2024-09-18T07:18:20.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.823+0000 7f7ef3707640 1 Processor -- start 2024-09-18T07:18:20.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.823+0000 7f7ef3707640 1 -- start start 2024-09-18T07:18:20.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ef3707640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eec138280 con 0x7f7eec072ea0 2024-09-18T07:18:20.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ef3707640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eec136140 con 0x7f7eec072ac0 2024-09-18T07:18:20.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ef3707640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eec134f60 con 0x7f7eec13f9f0 2024-09-18T07:18:20.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ef0ca2640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7eec072ea0 0x7f7eec132cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34182/0 (socket says 172.21.15.70:34182) 2024-09-18T07:18:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ef0ca2640 1 -- 172.21.15.70:0/2229993650 learned_addr learned my addr 172.21.15.70:0/2229993650 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1541404094 0 0) 0x7f7eec138280 con 0x7f7eec072ea0 2024-09-18T07:18:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ed0003520 con 0x7f7eec072ea0 2024-09-18T07:18:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2721847328 0 0) 0x7f7eec134f60 con 0x7f7eec13f9f0 2024-09-18T07:18:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7eec138280 con 0x7f7eec13f9f0 2024-09-18T07:18:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1587649910 0 0) 0x7f7ed0003520 con 0x7f7eec072ea0 2024-09-18T07:18:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7eec134f60 con 0x7f7eec072ea0 2024-09-18T07:18:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2990769528 0 0) 0x7f7eec138280 con 0x7f7eec13f9f0 2024-09-18T07:18:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7ed0003520 con 0x7f7eec13f9f0 2024-09-18T07:18:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7edc001a30 con 0x7f7eec072ea0 2024-09-18T07:18:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7ee4001a70 con 0x7f7eec13f9f0 2024-09-18T07:18:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2614982174 0 0) 0x7f7eec136140 con 0x7f7eec072ac0 2024-09-18T07:18:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7eec138280 con 0x7f7eec072ac0 2024-09-18T07:18:20.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1997951968 0 0) 0x7f7eec134f60 con 0x7f7eec072ea0 2024-09-18T07:18:20.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 >> v1:172.21.15.70:6790/0 conn(0x7f7eec13f9f0 legacy=0x7f7eec1e1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.824+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 >> v1:172.21.15.172:6789/0 conn(0x7f7eec072ac0 legacy=0x7f7eec1368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:20.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.825+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec134a30 con 0x7f7eec072ea0 2024-09-18T07:18:20.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.825+0000 7f7ef3707640 1 -- 172.21.15.70:0/2229993650 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7eec1333e0 con 0x7f7eec072ea0 2024-09-18T07:18:20.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.825+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7edc0038c0 con 0x7f7eec072ea0 2024-09-18T07:18:20.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.825+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f7edc003cd0 con 0x7f7eec072ea0 2024-09-18T07:18:20.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.825+0000 7f7ef3707640 1 -- 172.21.15.70:0/2229993650 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7eec1e2750 con 0x7f7eec072ea0 2024-09-18T07:18:20.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.826+0000 7f7ef3707640 1 -- 172.21.15.70:0/2229993650 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eec0b4da0 con 0x7f7eec072ea0 2024-09-18T07:18:20.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.826+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f7edc001bd0 con 0x7f7eec072ea0 2024-09-18T07:18:20.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.827+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f7edc096190 con 0x7f7eec072ea0 2024-09-18T07:18:20.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:20.831+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f7edc05ead0 con 0x7f7eec072ea0 2024-09-18T07:18:21.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:21.004+0000 7f7ef3707640 1 -- 172.21.15.70:0/2229993650 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7eec070e70 con 0x7f7eec072ea0 2024-09-18T07:18:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:21.005+0000 7f7ee1ffb640 1 -- 172.21.15.70:0/2229993650 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+14751 (unknown 675718476 0 1434885666) 0x7f7edc05e3d0 con 0x7f7eec072ea0 2024-09-18T07:18:21.008 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:18:21.008 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":26,"fsid":"1fa528ad-1630-4e95-9001-978a53f2a42c","created":"2024-09-18T07:13:53.234326+0000","modified":"2024-09-18T07:18:19.233067+0000","last_up_change":"2024-09-18T07:14:04.459974+0000","last_in_change":"2024-09-18T07:13:55.403239+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T07:14:05.461421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T07:14:11.323426+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephsqlite","create_time":"2024-09-18T07:14:20.410273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4a2c5fa-37b8-4776-bf81-f142248da0da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":1022808125}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":1022808125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":1022808125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":1022808125}]},"public_addr":"172.21.15.70:6808/1022808125","cluster_addr":"172.21.15.70:6809/1022808125","heartbeat_back_addr":"172.21.15.70:6811/1022808125","heartbeat_front_addr":"172.21.15.70:6810/1022808125","state":["exists","up"]},{"osd":1,"uuid":"3fab2225-9881-4caf-8142-41b857b9b99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":1678886977}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":1678886977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":1678886977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":1678886977}]},"public_addr":"172.21.15.70:6800/1678886977","cluster_addr":"172.21.15.70:6801/1678886977","heartbeat_back_addr":"172.21.15.70:6803/1678886977","heartbeat_front_addr":"172.21.15.70:6802/1678886977","state":["exists","up"]},{"osd":2,"uuid":"5d6764cd-e7a4-45df-9cfb-88d9e9a2675f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":3042930971}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":3042930971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":3042930971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":3042930971}]},"public_addr":"172.21.15.70:6812/3042930971","cluster_addr":"172.21.15.70:6813/3042930971","heartbeat_back_addr":"172.21.15.70:6815/3042930971","heartbeat_front_addr":"172.21.15.70:6814/3042930971","state":["exists","up"]},{"osd":3,"uuid":"80961d78-b7ae-416d-bb6f-a2ef724c6cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":202753903}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":202753903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":202753903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":202753903}]},"public_addr":"172.21.15.70:6804/202753903","cluster_addr":"172.21.15.70:6805/202753903","heartbeat_back_addr":"172.21.15.70:6807/202753903","heartbeat_front_addr":"172.21.15.70:6806/202753903","state":["exists","up"]},{"osd":4,"uuid":"ac72b675-4df6-4668-8461-3d0d8ac698b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6808","nonce":3405680289}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6809","nonce":3405680289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6811","nonce":3405680289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6810","nonce":3405680289}]},"public_addr":"172.21.15.172:6808/3405680289","cluster_addr":"172.21.15.172:6809/3405680289","heartbeat_back_addr":"172.21.15.172:6811/3405680289","heartbeat_front_addr":"172.21.15.172:6810/3405680289","state":["exists","up"]},{"osd":5,"uuid":"5949328d-408f-4b72-8e51-db0046fbb090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6804","nonce":3053548626}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6805","nonce":3053548626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6807","nonce":3053548626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6806","nonce":3053548626}]},"public_addr":"172.21.15.172:6804/3053548626","cluster_addr":"172.21.15.172:6805/3053548626","heartbeat_back_addr":"172.21.15.172:6807/3053548626","heartbeat_front_addr":"172.21.15.172:6806/3053548626","state":["exists","up"]},{"osd":6,"uuid":"84ce0e0b-9072-4ef7-9e55-f3d81283b3de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6812","nonce":371905845}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6813","nonce":371905845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6815","nonce":371905845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6814","nonce":371905845}]},"public_addr":"172.21.15.172:6812/371905845","cluster_addr":"172.21.15.172:6813/371905845","heartbeat_back_addr":"172.21.15.172:6815/371905845","heartbeat_front_addr":"172.21.15.172:6814/371905845","state":["exists","up"]},{"osd":7,"uuid":"96889def-b533-4738-bb6d-dd8eb99e1e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6800","nonce":2423240725}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6801","nonce":2423240725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6803","nonce":2423240725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.172:6802","nonce":2423240725}]},"public_addr":"172.21.15.172:6800/2423240725","cluster_addr":"172.21.15.172:6801/2423240725","heartbeat_back_addr":"172.21.15.172:6803/2423240725","heartbeat_front_addr":"172.21.15.172:6802/2423240725","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.866556+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.704725+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:02.409291+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.477089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:02.747153+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.980019+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.715388+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T07:14:01.500717+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1841697781":"2024-09-18T08:18:18.309618+0000","172.21.15.70:0/682334394":"2024-09-18T08:17:02.238650+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T07:18:21.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:21.007+0000 7f7ef3707640 1 -- 172.21.15.70:0/2229993650 >> v1:172.21.15.70:6816/1657990791 conn(0x7f7ed0079880 legacy=0x7f7ed007bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:21.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:21.007+0000 7f7ef3707640 1 -- 172.21.15.70:0/2229993650 >> v1:172.21.15.70:6789/0 conn(0x7f7eec072ea0 legacy=0x7f7eec132cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:21.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:21.007+0000 7f7ef3707640 1 -- 172.21.15.70:0/2229993650 shutdown_connections 2024-09-18T07:18:21.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:21.007+0000 7f7ef3707640 1 -- 172.21.15.70:0/2229993650 >> 172.21.15.70:0/2229993650 conn(0x7f7eec0b5790 msgr2=0x7f7eec143930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:21.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:21.007+0000 7f7ef3707640 1 -- 172.21.15.70:0/2229993650 shutdown_connections 2024-09-18T07:18:21.011 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:21.007+0000 7f7ef3707640 1 -- 172.21.15.70:0/2229993650 wait complete. 2024-09-18T07:18:22.021 INFO:tasks.ceph:Scrubbing osd.0 2024-09-18T07:18:22.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-18T07:18:22.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.134+0000 7fa146da9640 1 Processor -- start 2024-09-18T07:18:22.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.134+0000 7fa146da9640 1 -- start start 2024-09-18T07:18:22.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.134+0000 7fa146da9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1400aab30 con 0x7fa140140b60 2024-09-18T07:18:22.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.134+0000 7fa146da9640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1400aae10 con 0x7fa14013d610 2024-09-18T07:18:22.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.134+0000 7fa146da9640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa14013ff30 con 0x7fa140072b30 2024-09-18T07:18:22.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.134+0000 7fa144b45640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fa140072b30 0x7fa140140450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48118/0 (socket says 172.21.15.70:48118) 2024-09-18T07:18:22.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.134+0000 7fa144b45640 1 -- 172.21.15.70:0/2294516091 learned_addr learned my addr 172.21.15.70:0/2294516091 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:22.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.134+0000 7fa13ffff640 1 --1- 172.21.15.70:0/2294516091 >> v1:172.21.15.70:6789/0 conn(0x7fa140140b60 0x7fa14013cf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57536/0 (socket says 172.21.15.70:57536) 2024-09-18T07:18:22.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.135+0000 7fa13f7fe640 1 -- 172.21.15.70:0/2294516091 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1819530868 0 0) 0x7fa1400aab30 con 0x7fa140140b60 2024-09-18T07:18:22.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.135+0000 7fa13f7fe640 1 -- 172.21.15.70:0/2294516091 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa128003520 con 0x7fa140140b60 2024-09-18T07:18:22.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.135+0000 7fa13f7fe640 1 -- 172.21.15.70:0/2294516091 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1219689324 0 0) 0x7fa14013ff30 con 0x7fa140072b30 2024-09-18T07:18:22.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.135+0000 7fa13f7fe640 1 -- 172.21.15.70:0/2294516091 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1400aab30 con 0x7fa140072b30 2024-09-18T07:18:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.135+0000 7fa13f7fe640 1 -- 172.21.15.70:0/2294516091 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2354866809 0 0) 0x7fa1400aab30 con 0x7fa140072b30 2024-09-18T07:18:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.135+0000 7fa13f7fe640 1 -- 172.21.15.70:0/2294516091 >> v1:172.21.15.172:6789/0 conn(0x7fa14013d610 legacy=0x7fa14014b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.135+0000 7fa13f7fe640 1 -- 172.21.15.70:0/2294516091 >> v1:172.21.15.70:6789/0 conn(0x7fa140140b60 legacy=0x7fa14013cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.135+0000 7fa13f7fe640 1 -- 172.21.15.70:0/2294516091 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa140073f00 con 0x7fa140072b30 2024-09-18T07:18:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.135+0000 7fa13f7fe640 1 -- 172.21.15.70:0/2294516091 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fa130001670 con 0x7fa140072b30 2024-09-18T07:18:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.135+0000 7fa13f7fe640 1 -- 172.21.15.70:0/2294516091 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa130001810 con 0x7fa140072b30 2024-09-18T07:18:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.135+0000 7fa13f7fe640 1 -- 172.21.15.70:0/2294516091 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fa130001ca0 con 0x7fa140072b30 2024-09-18T07:18:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.136+0000 7fa146da9640 1 -- 172.21.15.70:0/2294516091 >> v1:172.21.15.70:6790/0 conn(0x7fa140072b30 legacy=0x7fa140140450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.136+0000 7fa146da9640 1 -- 172.21.15.70:0/2294516091 shutdown_connections 2024-09-18T07:18:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.136+0000 7fa146da9640 1 -- 172.21.15.70:0/2294516091 >> 172.21.15.70:0/2294516091 conn(0x7fa1400b5890 msgr2=0x7fa1400b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.136+0000 7fa146da9640 1 -- 172.21.15.70:0/2294516091 shutdown_connections 2024-09-18T07:18:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.136+0000 7fa146da9640 1 -- 172.21.15.70:0/2294516091 wait complete. 2024-09-18T07:18:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.137+0000 7fa146da9640 1 Processor -- start 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.137+0000 7fa146da9640 1 -- start start 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.137+0000 7fa146da9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa140130490 con 0x7fa14013d610 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.137+0000 7fa146da9640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa140142960 con 0x7fa140072b30 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.137+0000 7fa146da9640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa140142c40 con 0x7fa140140b60 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa14013d610 0x7fa14012f670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57546/0 (socket says 172.21.15.70:57546) 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13ffff640 1 -- 172.21.15.70:0/4150024875 learned_addr learned my addr 172.21.15.70:0/4150024875 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1962767008 0 0) 0x7fa140130490 con 0x7fa14013d610 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa12c003520 con 0x7fa14013d610 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4040875262 0 0) 0x7fa140142c40 con 0x7fa140140b60 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa140130490 con 0x7fa140140b60 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3443625715 0 0) 0x7fa140142960 con 0x7fa140072b30 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa140142c40 con 0x7fa140072b30 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 18623237 0 0) 0x7fa12c003520 con 0x7fa14013d610 2024-09-18T07:18:22.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa140142960 con 0x7fa14013d610 2024-09-18T07:18:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fa134001850 con 0x7fa14013d610 2024-09-18T07:18:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3995589899 0 0) 0x7fa140130490 con 0x7fa140140b60 2024-09-18T07:18:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa12c003520 con 0x7fa140140b60 2024-09-18T07:18:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fa138001c30 con 0x7fa140140b60 2024-09-18T07:18:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2277166488 0 0) 0x7fa140142960 con 0x7fa14013d610 2024-09-18T07:18:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 >> v1:172.21.15.70:6790/0 conn(0x7fa140140b60 legacy=0x7fa14012fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 >> v1:172.21.15.172:6789/0 conn(0x7fa140072b30 legacy=0x7fa140141310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa140143110 con 0x7fa14013d610 2024-09-18T07:18:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa146da9640 1 -- 172.21.15.70:0/4150024875 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa140141a20 con 0x7fa14013d610 2024-09-18T07:18:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa146da9640 1 -- 172.21.15.70:0/4150024875 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa140139df0 con 0x7fa14013d610 2024-09-18T07:18:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa1340047e0 con 0x7fa14013d610 2024-09-18T07:18:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.138+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fa134004b30 con 0x7fa14013d610 2024-09-18T07:18:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.139+0000 7fa146da9640 1 -- 172.21.15.70:0/4150024875 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fa120000f80 con 0x7fa14013d610 2024-09-18T07:18:22.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.140+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fa13401d900 con 0x7fa14013d610 2024-09-18T07:18:22.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.140+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7fa134096120 con 0x7fa14013d610 2024-09-18T07:18:22.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.140+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 --> v1:172.21.15.70:6808/1022808125 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa12c085250 con 0x7fa12c081680 2024-09-18T07:18:22.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.140+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (unknown 2370098492 0 0) 0x7fa1340964b0 con 0x7fa14013d610 2024-09-18T07:18:22.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.141+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== osd.0 v1:172.21.15.70:6808/1022808125 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fa12c085250 con 0x7fa12c081680 2024-09-18T07:18:22.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.154+0000 7fa146da9640 1 -- 172.21.15.70:0/4150024875 --> v1:172.21.15.70:6808/1022808125 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa120002cd0 con 0x7fa12c081680 2024-09-18T07:18:22.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.154+0000 7fa13d7fa640 1 -- 172.21.15.70:0/4150024875 <== osd.0 v1:172.21.15.70:6808/1022808125 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7fa120002cd0 con 0x7fa12c081680 2024-09-18T07:18:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-18T07:18:22.194 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T07:18:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-18T07:18:22.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.154+0000 7fa146da9640 1 -- 172.21.15.70:0/4150024875 >> v1:172.21.15.70:6808/1022808125 conn(0x7fa12c081680 legacy=0x7fa12c083aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.154+0000 7fa146da9640 1 -- 172.21.15.70:0/4150024875 >> v1:172.21.15.70:6816/1657990791 conn(0x7fa12c0797e0 legacy=0x7fa12c07bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.154+0000 7fa146da9640 1 -- 172.21.15.70:0/4150024875 >> v1:172.21.15.70:6789/0 conn(0x7fa14013d610 legacy=0x7fa14012f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.154+0000 7fa145346640 1 -- 172.21.15.70:0/4150024875 reap_dead start 2024-09-18T07:18:22.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.154+0000 7fa146da9640 1 -- 172.21.15.70:0/4150024875 shutdown_connections 2024-09-18T07:18:22.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.154+0000 7fa146da9640 1 -- 172.21.15.70:0/4150024875 >> 172.21.15.70:0/4150024875 conn(0x7fa1400b5890 msgr2=0x7fa1400b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:22.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.154+0000 7fa146da9640 1 -- 172.21.15.70:0/4150024875 shutdown_connections 2024-09-18T07:18:22.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.154+0000 7fa146da9640 1 -- 172.21.15.70:0/4150024875 wait complete. 2024-09-18T07:18:22.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-18T07:18:22.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.297+0000 7f3c263ec640 1 Processor -- start 2024-09-18T07:18:22.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.297+0000 7f3c263ec640 1 -- start start 2024-09-18T07:18:22.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c263ec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c20073680 con 0x7f3c20072ec0 2024-09-18T07:18:22.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c263ec640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c20071670 con 0x7f3c200732a0 2024-09-18T07:18:22.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c263ec640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c20072990 con 0x7f3c2013a5f0 2024-09-18T07:18:22.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1f7fe640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f3c2013a5f0 0x7f3c20071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48146/0 (socket says 172.21.15.70:48146) 2024-09-18T07:18:22.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3c20072ec0 0x7f3c20139ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57548/0 (socket says 172.21.15.70:57548) 2024-09-18T07:18:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1f7fe640 1 -- 172.21.15.70:0/1877167062 learned_addr learned my addr 172.21.15.70:0/1877167062 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1effd640 1 -- 172.21.15.70:0/1877167062 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3170348338 0 0) 0x7f3c10001110 con 0x7f3c20072ec0 2024-09-18T07:18:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1effd640 1 -- 172.21.15.70:0/1877167062 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c08003520 con 0x7f3c20072ec0 2024-09-18T07:18:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1effd640 1 -- 172.21.15.70:0/1877167062 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 204706887 0 0) 0x7f3c20072990 con 0x7f3c2013a5f0 2024-09-18T07:18:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1effd640 1 -- 172.21.15.70:0/1877167062 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c10001110 con 0x7f3c2013a5f0 2024-09-18T07:18:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1effd640 1 -- 172.21.15.70:0/1877167062 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4251294336 0 0) 0x7f3c20071670 con 0x7f3c200732a0 2024-09-18T07:18:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1effd640 1 -- 172.21.15.70:0/1877167062 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c20072990 con 0x7f3c200732a0 2024-09-18T07:18:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1effd640 1 -- 172.21.15.70:0/1877167062 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2598672215 0 0) 0x7f3c08003520 con 0x7f3c20072ec0 2024-09-18T07:18:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1effd640 1 -- 172.21.15.70:0/1877167062 >> v1:172.21.15.70:6790/0 conn(0x7f3c2013a5f0 legacy=0x7f3c20071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1effd640 1 -- 172.21.15.70:0/1877167062 >> v1:172.21.15.172:6789/0 conn(0x7f3c200732a0 legacy=0x7f3c20072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1effd640 1 -- 172.21.15.70:0/1877167062 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c20144bb0 con 0x7f3c20072ec0 2024-09-18T07:18:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.298+0000 7f3c1effd640 1 -- 172.21.15.70:0/1877167062 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f3c10001860 con 0x7f3c20072ec0 2024-09-18T07:18:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.299+0000 7f3c1effd640 1 -- 172.21.15.70:0/1877167062 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3c10001a00 con 0x7f3c20072ec0 2024-09-18T07:18:22.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.299+0000 7f3c1effd640 1 -- 172.21.15.70:0/1877167062 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f3c100047e0 con 0x7f3c20072ec0 2024-09-18T07:18:22.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.299+0000 7f3c263ec640 1 -- 172.21.15.70:0/1877167062 >> v1:172.21.15.70:6789/0 conn(0x7f3c20072ec0 legacy=0x7f3c20139ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.299+0000 7f3c263ec640 1 -- 172.21.15.70:0/1877167062 shutdown_connections 2024-09-18T07:18:22.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.299+0000 7f3c263ec640 1 -- 172.21.15.70:0/1877167062 >> 172.21.15.70:0/1877167062 conn(0x7f3c200b64d0 msgr2=0x7f3c200b68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:22.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.299+0000 7f3c263ec640 1 -- 172.21.15.70:0/1877167062 shutdown_connections 2024-09-18T07:18:22.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.299+0000 7f3c263ec640 1 -- 172.21.15.70:0/1877167062 wait complete. 2024-09-18T07:18:22.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.300+0000 7f3c263ec640 1 Processor -- start 2024-09-18T07:18:22.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.300+0000 7f3c263ec640 1 -- start start 2024-09-18T07:18:22.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.301+0000 7f3c263ec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c20145880 con 0x7f3c2013a5f0 2024-09-18T07:18:22.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.301+0000 7f3c263ec640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c20146aa0 con 0x7f3c20072ec0 2024-09-18T07:18:22.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.301+0000 7f3c263ec640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c20132eb0 con 0x7f3c200732a0 2024-09-18T07:18:22.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.301+0000 7f3c1f7fe640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f3c200732a0 0x7f3c2012ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48154/0 (socket says 172.21.15.70:48154) 2024-09-18T07:18:22.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.301+0000 7f3c1f7fe640 1 -- 172.21.15.70:0/3406456157 learned_addr learned my addr 172.21.15.70:0/3406456157 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:22.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.301+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2244281283 0 0) 0x7f3c20132eb0 con 0x7f3c200732a0 2024-09-18T07:18:22.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.301+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c0c003520 con 0x7f3c200732a0 2024-09-18T07:18:22.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.301+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2001241101 0 0) 0x7f3c20145880 con 0x7f3c2013a5f0 2024-09-18T07:18:22.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.301+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c20132eb0 con 0x7f3c2013a5f0 2024-09-18T07:18:22.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.301+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 157968263 0 0) 0x7f3c20146aa0 con 0x7f3c20072ec0 2024-09-18T07:18:22.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.301+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c20145880 con 0x7f3c20072ec0 2024-09-18T07:18:22.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.301+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 410955966 0 0) 0x7f3c0c003520 con 0x7f3c200732a0 2024-09-18T07:18:22.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.302+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3c20146aa0 con 0x7f3c200732a0 2024-09-18T07:18:22.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.302+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f3c14001890 con 0x7f3c200732a0 2024-09-18T07:18:22.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.302+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 619705982 0 0) 0x7f3c20132eb0 con 0x7f3c2013a5f0 2024-09-18T07:18:22.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.302+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3c0c003520 con 0x7f3c2013a5f0 2024-09-18T07:18:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.302+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f3c18001bf0 con 0x7f3c2013a5f0 2024-09-18T07:18:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.302+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 241010181 0 0) 0x7f3c20146aa0 con 0x7f3c200732a0 2024-09-18T07:18:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.302+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 >> v1:172.21.15.172:6789/0 conn(0x7f3c20072ec0 legacy=0x7f3c20146da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.302+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 >> v1:172.21.15.70:6789/0 conn(0x7f3c2013a5f0 legacy=0x7f3c201e4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.302+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c201d98b0 con 0x7f3c200732a0 2024-09-18T07:18:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.302+0000 7f3c263ec640 1 -- 172.21.15.70:0/3406456157 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3c20145da0 con 0x7f3c200732a0 2024-09-18T07:18:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.302+0000 7f3c263ec640 1 -- 172.21.15.70:0/3406456157 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f3c201474b0 con 0x7f3c200732a0 2024-09-18T07:18:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.302+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3c140047e0 con 0x7f3c200732a0 2024-09-18T07:18:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.302+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f3c14004b30 con 0x7f3c200732a0 2024-09-18T07:18:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.303+0000 7f3c263ec640 1 -- 172.21.15.70:0/3406456157 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c00005080 con 0x7f3c200732a0 2024-09-18T07:18:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.304+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f3c1401d900 con 0x7f3c200732a0 2024-09-18T07:18:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.304+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f3c140960b0 con 0x7f3c200732a0 2024-09-18T07:18:22.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.308+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f3c1405e910 con 0x7f3c200732a0 2024-09-18T07:18:22.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.484+0000 7f3c263ec640 1 -- 172.21.15.70:0/3406456157 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f3c00000b90 con 0x7f3c0c079880 2024-09-18T07:18:22.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.484+0000 7f3c1cff9640 1 -- 172.21.15.70:0/3406456157 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (unknown 2057319476 0 0) 0x7f3c00000b90 con 0x7f3c0c079880 2024-09-18T07:18:22.487 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 0 to deep-scrub 2024-09-18T07:18:22.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.486+0000 7f3c263ec640 1 -- 172.21.15.70:0/3406456157 >> v1:172.21.15.70:6816/1657990791 conn(0x7f3c0c079880 legacy=0x7f3c0c07bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.487+0000 7f3c263ec640 1 -- 172.21.15.70:0/3406456157 >> v1:172.21.15.70:6790/0 conn(0x7f3c200732a0 legacy=0x7f3c2012ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.487+0000 7f3c263ec640 1 -- 172.21.15.70:0/3406456157 shutdown_connections 2024-09-18T07:18:22.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.487+0000 7f3c263ec640 1 -- 172.21.15.70:0/3406456157 >> 172.21.15.70:0/3406456157 conn(0x7f3c200b64d0 msgr2=0x7f3c200702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:22.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.487+0000 7f3c263ec640 1 -- 172.21.15.70:0/3406456157 shutdown_connections 2024-09-18T07:18:22.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.487+0000 7f3c263ec640 1 -- 172.21.15.70:0/3406456157 wait complete. 2024-09-18T07:18:22.501 INFO:tasks.ceph:Scrubbing osd.1 2024-09-18T07:18:22.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-18T07:18:22.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.600+0000 7f16d8500640 1 Processor -- start 2024-09-18T07:18:22.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.600+0000 7f16d8500640 1 -- start start 2024-09-18T07:18:22.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.600+0000 7f16d8500640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d00aab30 con 0x7f16d0140b60 2024-09-18T07:18:22.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.600+0000 7f16d8500640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d00aae10 con 0x7f16d013d610 2024-09-18T07:18:22.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.600+0000 7f16d8500640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d013ff30 con 0x7f16d0072b30 2024-09-18T07:18:22.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.601+0000 7f16d629c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f16d0072b30 0x7f16d0140450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48164/0 (socket says 172.21.15.70:48164) 2024-09-18T07:18:22.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.601+0000 7f16d629c640 1 -- 172.21.15.70:0/2164392724 learned_addr learned my addr 172.21.15.70:0/2164392724 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:22.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.601+0000 7f16d529a640 1 -- 172.21.15.70:0/2164392724 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2460978916 0 0) 0x7f16d013ff30 con 0x7f16d0072b30 2024-09-18T07:18:22.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.601+0000 7f16d529a640 1 -- 172.21.15.70:0/2164392724 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16bc003520 con 0x7f16d0072b30 2024-09-18T07:18:22.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.601+0000 7f16d529a640 1 -- 172.21.15.70:0/2164392724 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2898117441 0 0) 0x7f16d00aab30 con 0x7f16d0140b60 2024-09-18T07:18:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.601+0000 7f16d529a640 1 -- 172.21.15.70:0/2164392724 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16d013ff30 con 0x7f16d0140b60 2024-09-18T07:18:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.601+0000 7f16d529a640 1 -- 172.21.15.70:0/2164392724 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 200710125 0 0) 0x7f16bc003520 con 0x7f16d0072b30 2024-09-18T07:18:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.601+0000 7f16d529a640 1 -- 172.21.15.70:0/2164392724 >> v1:172.21.15.172:6789/0 conn(0x7f16d013d610 legacy=0x7f16d014b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.601+0000 7f16d529a640 1 -- 172.21.15.70:0/2164392724 >> v1:172.21.15.70:6789/0 conn(0x7f16d0140b60 legacy=0x7f16d013cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.601+0000 7f16d529a640 1 -- 172.21.15.70:0/2164392724 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d0073f00 con 0x7f16d0072b30 2024-09-18T07:18:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.601+0000 7f16d529a640 1 -- 172.21.15.70:0/2164392724 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f16cc001690 con 0x7f16d0072b30 2024-09-18T07:18:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.601+0000 7f16d529a640 1 -- 172.21.15.70:0/2164392724 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f16cc0018b0 con 0x7f16d0072b30 2024-09-18T07:18:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.601+0000 7f16d529a640 1 -- 172.21.15.70:0/2164392724 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f16cc001d40 con 0x7f16d0072b30 2024-09-18T07:18:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.602+0000 7f16d8500640 1 -- 172.21.15.70:0/2164392724 >> v1:172.21.15.70:6790/0 conn(0x7f16d0072b30 legacy=0x7f16d0140450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.602+0000 7f16d8500640 1 -- 172.21.15.70:0/2164392724 shutdown_connections 2024-09-18T07:18:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.602+0000 7f16d8500640 1 -- 172.21.15.70:0/2164392724 >> 172.21.15.70:0/2164392724 conn(0x7f16d00b5890 msgr2=0x7f16d00b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.602+0000 7f16d8500640 1 -- 172.21.15.70:0/2164392724 shutdown_connections 2024-09-18T07:18:22.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.602+0000 7f16d8500640 1 -- 172.21.15.70:0/2164392724 wait complete. 2024-09-18T07:18:22.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.603+0000 7f16d8500640 1 Processor -- start 2024-09-18T07:18:22.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.603+0000 7f16d8500640 1 -- start start 2024-09-18T07:18:22.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.603+0000 7f16d8500640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d00afbb0 con 0x7f16d0140b60 2024-09-18T07:18:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.603+0000 7f16d8500640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d013a040 con 0x7f16d013d610 2024-09-18T07:18:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.603+0000 7f16d8500640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d00ade00 con 0x7f16d0072b30 2024-09-18T07:18:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.603+0000 7f16d629c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f16d0072b30 0x7f16d00ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48170/0 (socket says 172.21.15.70:48170) 2024-09-18T07:18:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.603+0000 7f16d629c640 1 -- 172.21.15.70:0/2235473996 learned_addr learned my addr 172.21.15.70:0/2235473996 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3516232127 0 0) 0x7f16d00afbb0 con 0x7f16d0140b60 2024-09-18T07:18:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16b4005730 con 0x7f16d0140b60 2024-09-18T07:18:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 609569004 0 0) 0x7f16d00ade00 con 0x7f16d0072b30 2024-09-18T07:18:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16d00afbb0 con 0x7f16d0072b30 2024-09-18T07:18:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3233583502 0 0) 0x7f16d013a040 con 0x7f16d013d610 2024-09-18T07:18:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16d00ade00 con 0x7f16d013d610 2024-09-18T07:18:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1791193601 0 0) 0x7f16b4005730 con 0x7f16d0140b60 2024-09-18T07:18:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f16d013a040 con 0x7f16d0140b60 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2936453471 0 0) 0x7f16d00afbb0 con 0x7f16d0072b30 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f16b4005730 con 0x7f16d0072b30 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f16c8001c30 con 0x7f16d0140b60 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f16cc001c00 con 0x7f16d0072b30 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2989332837 0 0) 0x7f16d013a040 con 0x7f16d0140b60 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 >> v1:172.21.15.70:6790/0 conn(0x7f16d0072b30 legacy=0x7f16d00ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 >> v1:172.21.15.172:6789/0 conn(0x7f16d013d610 legacy=0x7f16d00ab230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d00adfd0 con 0x7f16d0140b60 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16d8500640 1 -- 172.21.15.70:0/2235473996 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f16d00b0fb0 con 0x7f16d0140b60 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16d8500640 1 -- 172.21.15.70:0/2235473996 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f16d00b0800 con 0x7f16d0140b60 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f16c8003970 con 0x7f16d0140b60 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.604+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f16c8003cc0 con 0x7f16d0140b60 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.605+0000 7f16d8500640 1 -- 172.21.15.70:0/2235473996 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f16b8000f80 con 0x7f16d0140b60 2024-09-18T07:18:22.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.606+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f16c801ca90 con 0x7f16d0140b60 2024-09-18T07:18:22.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.606+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f16c8095370 con 0x7f16d0140b60 2024-09-18T07:18:22.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.606+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 --> v1:172.21.15.70:6800/1678886977 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16b40874a0 con 0x7f16b4083830 2024-09-18T07:18:22.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.606+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (unknown 2370098492 0 0) 0x7f16c8095700 con 0x7f16d0140b60 2024-09-18T07:18:22.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.607+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== osd.1 v1:172.21.15.70:6800/1678886977 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f16b40874a0 con 0x7f16b4083830 2024-09-18T07:18:22.622 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.619+0000 7f16d8500640 1 -- 172.21.15.70:0/2235473996 --> v1:172.21.15.70:6800/1678886977 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f16b8002cd0 con 0x7f16b4083830 2024-09-18T07:18:22.622 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.620+0000 7f16c6ffd640 1 -- 172.21.15.70:0/2235473996 <== osd.1 v1:172.21.15.70:6800/1678886977 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f16b8002cd0 con 0x7f16b4083830 2024-09-18T07:18:22.623 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-18T07:18:22.623 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T07:18:22.623 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-18T07:18:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.620+0000 7f16d8500640 1 -- 172.21.15.70:0/2235473996 >> v1:172.21.15.70:6800/1678886977 conn(0x7f16b4083830 legacy=0x7f16b4085c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.620+0000 7f16d8500640 1 -- 172.21.15.70:0/2235473996 >> v1:172.21.15.70:6816/1657990791 conn(0x7f16b407ba20 legacy=0x7f16b407dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.620+0000 7f16d8500640 1 -- 172.21.15.70:0/2235473996 >> v1:172.21.15.70:6789/0 conn(0x7f16d0140b60 legacy=0x7f16d00ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.620+0000 7f16d6a9d640 1 -- 172.21.15.70:0/2235473996 reap_dead start 2024-09-18T07:18:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.620+0000 7f16d8500640 1 -- 172.21.15.70:0/2235473996 shutdown_connections 2024-09-18T07:18:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.620+0000 7f16d8500640 1 -- 172.21.15.70:0/2235473996 >> 172.21.15.70:0/2235473996 conn(0x7f16d00b5890 msgr2=0x7f16d013bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.620+0000 7f16d8500640 1 -- 172.21.15.70:0/2235473996 shutdown_connections 2024-09-18T07:18:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.620+0000 7f16d8500640 1 -- 172.21.15.70:0/2235473996 wait complete. 2024-09-18T07:18:22.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-18T07:18:22.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.729+0000 7ff982cb8640 1 Processor -- start 2024-09-18T07:18:22.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.729+0000 7ff982cb8640 1 -- start start 2024-09-18T07:18:22.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff982cb8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97c13fb80 con 0x7ff97c072d70 2024-09-18T07:18:22.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff982cb8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97c0aaaa0 con 0x7ff97c072990 2024-09-18T07:18:22.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff982cb8640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97c0aad80 con 0x7ff97c140980 2024-09-18T07:18:22.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff97c072d70 0x7ff97c13ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57584/0 (socket says 172.21.15.70:57584) 2024-09-18T07:18:22.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97bfff640 1 -- 172.21.15.70:0/3885384413 learned_addr learned my addr 172.21.15.70:0/3885384413 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:22.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff981255640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7ff97c140980 0x7ff97c13f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48174/0 (socket says 172.21.15.70:48174) 2024-09-18T07:18:22.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97b7fe640 1 -- 172.21.15.70:0/3885384413 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 131221992 0 0) 0x7ff97c0aad80 con 0x7ff97c140980 2024-09-18T07:18:22.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97b7fe640 1 -- 172.21.15.70:0/3885384413 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff964003440 con 0x7ff97c140980 2024-09-18T07:18:22.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97b7fe640 1 -- 172.21.15.70:0/3885384413 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3264657097 0 0) 0x7ff97c13fb80 con 0x7ff97c072d70 2024-09-18T07:18:22.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97b7fe640 1 -- 172.21.15.70:0/3885384413 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff97c0aad80 con 0x7ff97c072d70 2024-09-18T07:18:22.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97b7fe640 1 -- 172.21.15.70:0/3885384413 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 848078630 0 0) 0x7ff97c0aaaa0 con 0x7ff97c072990 2024-09-18T07:18:22.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97b7fe640 1 -- 172.21.15.70:0/3885384413 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff97c13fb80 con 0x7ff97c072990 2024-09-18T07:18:22.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97b7fe640 1 -- 172.21.15.70:0/3885384413 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3993259122 0 0) 0x7ff97c0aad80 con 0x7ff97c072d70 2024-09-18T07:18:22.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97b7fe640 1 -- 172.21.15.70:0/3885384413 >> v1:172.21.15.70:6790/0 conn(0x7ff97c140980 legacy=0x7ff97c13f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97b7fe640 1 -- 172.21.15.70:0/3885384413 >> v1:172.21.15.172:6789/0 conn(0x7ff97c072990 legacy=0x7ff97c140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97b7fe640 1 -- 172.21.15.70:0/3885384413 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c074410 con 0x7ff97c072d70 2024-09-18T07:18:22.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97b7fe640 1 -- 172.21.15.70:0/3885384413 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff9700019f0 con 0x7ff97c072d70 2024-09-18T07:18:22.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97b7fe640 1 -- 172.21.15.70:0/3885384413 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff970001c00 con 0x7ff97c072d70 2024-09-18T07:18:22.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.730+0000 7ff97b7fe640 1 -- 172.21.15.70:0/3885384413 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff970001fd0 con 0x7ff97c072d70 2024-09-18T07:18:22.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.731+0000 7ff982cb8640 1 -- 172.21.15.70:0/3885384413 >> v1:172.21.15.70:6789/0 conn(0x7ff97c072d70 legacy=0x7ff97c13ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.731+0000 7ff982cb8640 1 -- 172.21.15.70:0/3885384413 shutdown_connections 2024-09-18T07:18:22.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.731+0000 7ff982cb8640 1 -- 172.21.15.70:0/3885384413 >> 172.21.15.70:0/3885384413 conn(0x7ff97c076bb0 msgr2=0x7ff97c0b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:22.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.731+0000 7ff982cb8640 1 -- 172.21.15.70:0/3885384413 shutdown_connections 2024-09-18T07:18:22.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.731+0000 7ff982cb8640 1 -- 172.21.15.70:0/3885384413 wait complete. 2024-09-18T07:18:22.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.732+0000 7ff982cb8640 1 Processor -- start 2024-09-18T07:18:22.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.732+0000 7ff982cb8640 1 -- start start 2024-09-18T07:18:22.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff982cb8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97c0ad6c0 con 0x7ff97c072d70 2024-09-18T07:18:22.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff982cb8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97c13b300 con 0x7ff97c072990 2024-09-18T07:18:22.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff982cb8640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97c0ad2e0 con 0x7ff97c140980 2024-09-18T07:18:22.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff97bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff97c072d70 0x7ff97c139b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57596/0 (socket says 172.21.15.70:57596) 2024-09-18T07:18:22.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff97bfff640 1 -- 172.21.15.70:0/4181553947 learned_addr learned my addr 172.21.15.70:0/4181553947 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:22.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525908734 0 0) 0x7ff97c0ad2e0 con 0x7ff97c140980 2024-09-18T07:18:22.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff968003520 con 0x7ff97c140980 2024-09-18T07:18:22.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3205033292 0 0) 0x7ff97c0ad6c0 con 0x7ff97c072d70 2024-09-18T07:18:22.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff97c0ad2e0 con 0x7ff97c072d70 2024-09-18T07:18:22.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 521952098 0 0) 0x7ff97c13b300 con 0x7ff97c072990 2024-09-18T07:18:22.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff97c0ad6c0 con 0x7ff97c072990 2024-09-18T07:18:22.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2346905195 0 0) 0x7ff968003520 con 0x7ff97c140980 2024-09-18T07:18:22.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff97c13b300 con 0x7ff97c140980 2024-09-18T07:18:22.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff974001790 con 0x7ff97c140980 2024-09-18T07:18:22.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1804772876 0 0) 0x7ff97c0ad2e0 con 0x7ff97c072d70 2024-09-18T07:18:22.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff968003520 con 0x7ff97c072d70 2024-09-18T07:18:22.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff9700018b0 con 0x7ff97c072d70 2024-09-18T07:18:22.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2437932652 0 0) 0x7ff97c13b300 con 0x7ff97c140980 2024-09-18T07:18:22.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 >> v1:172.21.15.172:6789/0 conn(0x7ff97c072990 legacy=0x7ff97c0ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 >> v1:172.21.15.70:6789/0 conn(0x7ff97c072d70 legacy=0x7ff97c139b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c0ab0f0 con 0x7ff97c140980 2024-09-18T07:18:22.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff982cb8640 1 -- 172.21.15.70:0/4181553947 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff97c0acdb0 con 0x7ff97c140980 2024-09-18T07:18:22.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff982cb8640 1 -- 172.21.15.70:0/4181553947 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7ff97c13aa40 con 0x7ff97c140980 2024-09-18T07:18:22.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.733+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff974001cc0 con 0x7ff97c140980 2024-09-18T07:18:22.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.734+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7ff974004980 con 0x7ff97c140980 2024-09-18T07:18:22.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.734+0000 7ff982cb8640 1 -- 172.21.15.70:0/4181553947 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff97c0ad4b0 con 0x7ff97c140980 2024-09-18T07:18:22.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.735+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7ff974004980 con 0x7ff97c140980 2024-09-18T07:18:22.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.736+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7ff974095d60 con 0x7ff97c140980 2024-09-18T07:18:22.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.740+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7ff97405e6a0 con 0x7ff97c140980 2024-09-18T07:18:22.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.913+0000 7ff982cb8640 1 -- 172.21.15.70:0/4181553947 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7ff97c0adbe0 con 0x7ff968079540 2024-09-18T07:18:22.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.914+0000 7ff9797fa640 1 -- 172.21.15.70:0/4181553947 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (unknown 489605429 0 0) 0x7ff96c005780 con 0x7ff968079540 2024-09-18T07:18:22.917 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 1 to deep-scrub 2024-09-18T07:18:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.916+0000 7ff982cb8640 1 -- 172.21.15.70:0/4181553947 >> v1:172.21.15.70:6816/1657990791 conn(0x7ff968079540 legacy=0x7ff96807ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.916+0000 7ff982cb8640 1 -- 172.21.15.70:0/4181553947 >> v1:172.21.15.70:6790/0 conn(0x7ff97c140980 legacy=0x7ff97c1e00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.916+0000 7ff982cb8640 1 -- 172.21.15.70:0/4181553947 shutdown_connections 2024-09-18T07:18:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.916+0000 7ff982cb8640 1 -- 172.21.15.70:0/4181553947 >> 172.21.15.70:0/4181553947 conn(0x7ff97c076bb0 msgr2=0x7ff97c071050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.916+0000 7ff982cb8640 1 -- 172.21.15.70:0/4181553947 shutdown_connections 2024-09-18T07:18:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:22.916+0000 7ff982cb8640 1 -- 172.21.15.70:0/4181553947 wait complete. 2024-09-18T07:18:22.929 INFO:tasks.ceph:Scrubbing osd.2 2024-09-18T07:18:22.930 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-18T07:18:23.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.028+0000 7f60ccd3d640 1 Processor -- start 2024-09-18T07:18:23.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.028+0000 7f60ccd3d640 1 -- start start 2024-09-18T07:18:23.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60ccd3d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c80ab340 con 0x7f60c8074330 2024-09-18T07:18:23.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60ccd3d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c8144ed0 con 0x7f60c81418b0 2024-09-18T07:18:23.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60ccd3d640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c80ab540 con 0x7f60c8073f50 2024-09-18T07:18:23.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60bffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f60c8074330 0x7f60c80aac30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57604/0 (socket says 172.21.15.70:57604) 2024-09-18T07:18:23.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60c659c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f60c8073f50 0x7f60c813b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48188/0 (socket says 172.21.15.70:48188) 2024-09-18T07:18:23.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60bffff640 1 -- 172.21.15.70:0/2176751521 learned_addr learned my addr 172.21.15.70:0/2176751521 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:23.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60c5d9b640 1 -- 172.21.15.70:0/2176751521 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 883379655 0 0) 0x7f60c80ab340 con 0x7f60c8074330 2024-09-18T07:18:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60c5d9b640 1 -- 172.21.15.70:0/2176751521 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60b4003520 con 0x7f60c8074330 2024-09-18T07:18:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60c5d9b640 1 -- 172.21.15.70:0/2176751521 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 702187770 0 0) 0x7f60c80ab540 con 0x7f60c8073f50 2024-09-18T07:18:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60c5d9b640 1 -- 172.21.15.70:0/2176751521 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60c80ab340 con 0x7f60c8073f50 2024-09-18T07:18:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60c5d9b640 1 -- 172.21.15.70:0/2176751521 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2484375881 0 0) 0x7f60b4003520 con 0x7f60c8074330 2024-09-18T07:18:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60c5d9b640 1 -- 172.21.15.70:0/2176751521 >> v1:172.21.15.70:6790/0 conn(0x7f60c8073f50 legacy=0x7f60c813b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60c5d9b640 1 -- 172.21.15.70:0/2176751521 >> v1:172.21.15.172:6789/0 conn(0x7f60c81418b0 legacy=0x7f60c81453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60c5d9b640 1 -- 172.21.15.70:0/2176751521 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c8145ac0 con 0x7f60c8074330 2024-09-18T07:18:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60c5d9b640 1 -- 172.21.15.70:0/2176751521 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f60b0001690 con 0x7f60c8074330 2024-09-18T07:18:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60c5d9b640 1 -- 172.21.15.70:0/2176751521 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f60b0001920 con 0x7f60c8074330 2024-09-18T07:18:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.029+0000 7f60c5d9b640 1 -- 172.21.15.70:0/2176751521 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f60b0001cf0 con 0x7f60c8074330 2024-09-18T07:18:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.030+0000 7f60ccd3d640 1 -- 172.21.15.70:0/2176751521 >> v1:172.21.15.70:6789/0 conn(0x7f60c8074330 legacy=0x7f60c80aac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.030+0000 7f60ccd3d640 1 -- 172.21.15.70:0/2176751521 shutdown_connections 2024-09-18T07:18:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.030+0000 7f60ccd3d640 1 -- 172.21.15.70:0/2176751521 >> 172.21.15.70:0/2176751521 conn(0x7f60c80b5890 msgr2=0x7f60c80b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.030+0000 7f60ccd3d640 1 -- 172.21.15.70:0/2176751521 shutdown_connections 2024-09-18T07:18:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.030+0000 7f60ccd3d640 1 -- 172.21.15.70:0/2176751521 wait complete. 2024-09-18T07:18:23.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.031+0000 7f60ccd3d640 1 Processor -- start 2024-09-18T07:18:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.031+0000 7f60ccd3d640 1 -- start start 2024-09-18T07:18:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60ccd3d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c81334e0 con 0x7f60c8073f50 2024-09-18T07:18:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60ccd3d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c81d6bb0 con 0x7f60c8074330 2024-09-18T07:18:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60ccd3d640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c813ba20 con 0x7f60c81418b0 2024-09-18T07:18:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60c659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f60c8073f50 0x7f60c81d64a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57606/0 (socket says 172.21.15.70:57606) 2024-09-18T07:18:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60c659c640 1 -- 172.21.15.70:0/3930113483 learned_addr learned my addr 172.21.15.70:0/3930113483 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3905341163 0 0) 0x7f60c81334e0 con 0x7f60c8073f50 2024-09-18T07:18:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60a8003520 con 0x7f60c8073f50 2024-09-18T07:18:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4132463756 0 0) 0x7f60c813ba20 con 0x7f60c81418b0 2024-09-18T07:18:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60c81334e0 con 0x7f60c81418b0 2024-09-18T07:18:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 62011375 0 0) 0x7f60a8003520 con 0x7f60c8073f50 2024-09-18T07:18:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f60c813ba20 con 0x7f60c8073f50 2024-09-18T07:18:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 178408157 0 0) 0x7f60c81d6bb0 con 0x7f60c8074330 2024-09-18T07:18:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60a8003520 con 0x7f60c8074330 2024-09-18T07:18:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 576647756 0 0) 0x7f60c81334e0 con 0x7f60c81418b0 2024-09-18T07:18:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f60c81d6bb0 con 0x7f60c81418b0 2024-09-18T07:18:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f60b8001200 con 0x7f60c8073f50 2024-09-18T07:18:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f60c0001a90 con 0x7f60c81418b0 2024-09-18T07:18:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2042771592 0 0) 0x7f60c81d6bb0 con 0x7f60c81418b0 2024-09-18T07:18:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 >> v1:172.21.15.172:6789/0 conn(0x7f60c8074330 legacy=0x7f60c81326c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 >> v1:172.21.15.70:6789/0 conn(0x7f60c8073f50 legacy=0x7f60c81d64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c813b520 con 0x7f60c81418b0 2024-09-18T07:18:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60ccd3d640 1 -- 172.21.15.70:0/3930113483 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f60c813bee0 con 0x7f60c81418b0 2024-09-18T07:18:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60ccd3d640 1 -- 172.21.15.70:0/3930113483 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f60c8135f90 con 0x7f60c81418b0 2024-09-18T07:18:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f60c00020a0 con 0x7f60c81418b0 2024-09-18T07:18:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.032+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f60c0003b30 con 0x7f60c81418b0 2024-09-18T07:18:23.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.033+0000 7f60ccd3d640 1 -- 172.21.15.70:0/3930113483 --> v1:172.21.15.70:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f60ac000f80 con 0x7f60c81418b0 2024-09-18T07:18:23.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.034+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f60c001c900 con 0x7f60c81418b0 2024-09-18T07:18:23.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.034+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f60c0095110 con 0x7f60c81418b0 2024-09-18T07:18:23.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.035+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 --> v1:172.21.15.70:6812/3042930971 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60a80852f0 con 0x7f60a8081680 2024-09-18T07:18:23.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.035+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (unknown 2370098492 0 0) 0x7f60c0095520 con 0x7f60c81418b0 2024-09-18T07:18:23.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.035+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== osd.2 v1:172.21.15.70:6812/3042930971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f60a80852f0 con 0x7f60a8081680 2024-09-18T07:18:23.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.048+0000 7f60ccd3d640 1 -- 172.21.15.70:0/3930113483 --> v1:172.21.15.70:6812/3042930971 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f60ac002cd0 con 0x7f60a8081680 2024-09-18T07:18:23.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.048+0000 7f60beffd640 1 -- 172.21.15.70:0/3930113483 <== osd.2 v1:172.21.15.70:6812/3042930971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f60ac002cd0 con 0x7f60a8081680 2024-09-18T07:18:23.051 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-18T07:18:23.051 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T07:18:23.051 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-18T07:18:23.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.048+0000 7f60ccd3d640 1 -- 172.21.15.70:0/3930113483 >> v1:172.21.15.70:6812/3042930971 conn(0x7f60a8081680 legacy=0x7f60a8083aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.048+0000 7f60ccd3d640 1 -- 172.21.15.70:0/3930113483 >> v1:172.21.15.70:6816/1657990791 conn(0x7f60a8079870 legacy=0x7f60a807bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.048+0000 7f60ccd3d640 1 -- 172.21.15.70:0/3930113483 >> v1:172.21.15.70:6790/0 conn(0x7f60c81418b0 legacy=0x7f60c8132dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.048+0000 7f60c6d9d640 1 -- 172.21.15.70:0/3930113483 reap_dead start 2024-09-18T07:18:23.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.048+0000 7f60ccd3d640 1 -- 172.21.15.70:0/3930113483 shutdown_connections 2024-09-18T07:18:23.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.048+0000 7f60ccd3d640 1 -- 172.21.15.70:0/3930113483 >> 172.21.15.70:0/3930113483 conn(0x7f60c80b5890 msgr2=0x7f60c81453b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:23.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.048+0000 7f60ccd3d640 1 -- 172.21.15.70:0/3930113483 shutdown_connections 2024-09-18T07:18:23.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.048+0000 7f60ccd3d640 1 -- 172.21.15.70:0/3930113483 wait complete. 2024-09-18T07:18:23.061 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-18T07:18:23.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.158+0000 7fc57c6ed640 1 Processor -- start 2024-09-18T07:18:23.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.158+0000 7fc57c6ed640 1 -- start start 2024-09-18T07:18:23.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc57c6ed640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5741413d0 con 0x7fc57406d8d0 2024-09-18T07:18:23.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc57c6ed640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc574070f20 con 0x7fc5740b1350 2024-09-18T07:18:23.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc57c6ed640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5740b0520 con 0x7fc574143a40 2024-09-18T07:18:23.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc57a489640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc57406d8d0 0x7fc574140cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57622/0 (socket says 172.21.15.70:57622) 2024-09-18T07:18:23.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc57a489640 1 -- 172.21.15.70:0/1546865043 learned_addr learned my addr 172.21.15.70:0/1546865043 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:23.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc579487640 1 -- 172.21.15.70:0/1546865043 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 714740082 0 0) 0x7fc5741413d0 con 0x7fc57406d8d0 2024-09-18T07:18:23.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc579487640 1 -- 172.21.15.70:0/1546865043 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc560003520 con 0x7fc57406d8d0 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc579487640 1 -- 172.21.15.70:0/1546865043 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2894231984 0 0) 0x7fc5740b0520 con 0x7fc574143a40 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc579487640 1 -- 172.21.15.70:0/1546865043 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5741413d0 con 0x7fc574143a40 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc579487640 1 -- 172.21.15.70:0/1546865043 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3859981565 0 0) 0x7fc560003520 con 0x7fc57406d8d0 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc579487640 1 -- 172.21.15.70:0/1546865043 >> v1:172.21.15.70:6790/0 conn(0x7fc574143a40 legacy=0x7fc57414f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc579487640 1 -- 172.21.15.70:0/1546865043 >> v1:172.21.15.172:6789/0 conn(0x7fc5740b1350 legacy=0x7fc574147450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc579487640 1 -- 172.21.15.70:0/1546865043 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5740b19b0 con 0x7fc57406d8d0 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.159+0000 7fc579487640 1 -- 172.21.15.70:0/1546865043 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc570001a10 con 0x7fc57406d8d0 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.160+0000 7fc579487640 1 -- 172.21.15.70:0/1546865043 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc570001c30 con 0x7fc57406d8d0 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.160+0000 7fc579487640 1 -- 172.21.15.70:0/1546865043 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc5700020c0 con 0x7fc57406d8d0 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.160+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1546865043 >> v1:172.21.15.70:6789/0 conn(0x7fc57406d8d0 legacy=0x7fc574140cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.160+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1546865043 shutdown_connections 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.160+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1546865043 >> 172.21.15.70:0/1546865043 conn(0x7fc574076130 msgr2=0x7fc574076510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.160+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1546865043 shutdown_connections 2024-09-18T07:18:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.160+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1546865043 wait complete. 2024-09-18T07:18:23.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.161+0000 7fc57c6ed640 1 Processor -- start 2024-09-18T07:18:23.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.161+0000 7fc57c6ed640 1 -- start start 2024-09-18T07:18:23.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc57c6ed640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc57413f5f0 con 0x7fc574143a40 2024-09-18T07:18:23.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc57c6ed640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc574121c30 con 0x7fc5740b1350 2024-09-18T07:18:23.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc57c6ed640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5741d99e0 con 0x7fc57406d8d0 2024-09-18T07:18:23.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc57a489640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fc57406d8d0 0x7fc57411fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48208/0 (socket says 172.21.15.70:48208) 2024-09-18T07:18:23.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc57a489640 1 -- 172.21.15.70:0/1877263850 learned_addr learned my addr 172.21.15.70:0/1877263850 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:23.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3876432973 0 0) 0x7fc5741d99e0 con 0x7fc57406d8d0 2024-09-18T07:18:23.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc558003520 con 0x7fc57406d8d0 2024-09-18T07:18:23.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1426956309 0 0) 0x7fc57413f5f0 con 0x7fc574143a40 2024-09-18T07:18:23.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5741d99e0 con 0x7fc574143a40 2024-09-18T07:18:23.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2104070721 0 0) 0x7fc574121c30 con 0x7fc5740b1350 2024-09-18T07:18:23.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc57413f5f0 con 0x7fc5740b1350 2024-09-18T07:18:23.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3204471681 0 0) 0x7fc558003520 con 0x7fc57406d8d0 2024-09-18T07:18:23.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc574121c30 con 0x7fc57406d8d0 2024-09-18T07:18:23.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc570002040 con 0x7fc57406d8d0 2024-09-18T07:18:23.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 192370036 0 0) 0x7fc5741d99e0 con 0x7fc574143a40 2024-09-18T07:18:23.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc558003520 con 0x7fc574143a40 2024-09-18T07:18:23.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.162+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc56c001210 con 0x7fc574143a40 2024-09-18T07:18:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.163+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4021895698 0 0) 0x7fc574121c30 con 0x7fc57406d8d0 2024-09-18T07:18:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.163+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 >> v1:172.21.15.172:6789/0 conn(0x7fc5740b1350 legacy=0x7fc57413e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.163+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 >> v1:172.21.15.70:6789/0 conn(0x7fc574143a40 legacy=0x7fc57413eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.163+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc574120160 con 0x7fc57406d8d0 2024-09-18T07:18:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.163+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1877263850 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc5741dbec0 con 0x7fc57406d8d0 2024-09-18T07:18:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.163+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1877263850 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fc574120a40 con 0x7fc57406d8d0 2024-09-18T07:18:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.163+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc570001570 con 0x7fc57406d8d0 2024-09-18T07:18:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.163+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fc570004fb0 con 0x7fc57406d8d0 2024-09-18T07:18:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.164+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1877263850 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc55c005080 con 0x7fc57406d8d0 2024-09-18T07:18:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.164+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fc570005150 con 0x7fc57406d8d0 2024-09-18T07:18:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.165+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7fc570096510 con 0x7fc57406d8d0 2024-09-18T07:18:23.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.169+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7fc57005ed50 con 0x7fc57406d8d0 2024-09-18T07:18:23.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.347+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1877263850 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fc55c000b90 con 0x7fc558079870 2024-09-18T07:18:23.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.347+0000 7fc56affd640 1 -- 172.21.15.70:0/1877263850 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (unknown 3049115190 0 0) 0x7fc55c000b90 con 0x7fc558079870 2024-09-18T07:18:23.350 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 2 to deep-scrub 2024-09-18T07:18:23.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.350+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1877263850 >> v1:172.21.15.70:6816/1657990791 conn(0x7fc558079870 legacy=0x7fc55807bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.350+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1877263850 >> v1:172.21.15.70:6790/0 conn(0x7fc57406d8d0 legacy=0x7fc57411fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.350+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1877263850 shutdown_connections 2024-09-18T07:18:23.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.350+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1877263850 >> 172.21.15.70:0/1877263850 conn(0x7fc574076130 msgr2=0x7fc574145330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:23.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.350+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1877263850 shutdown_connections 2024-09-18T07:18:23.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.350+0000 7fc57c6ed640 1 -- 172.21.15.70:0/1877263850 wait complete. 2024-09-18T07:18:23.363 INFO:tasks.ceph:Scrubbing osd.3 2024-09-18T07:18:23.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-18T07:18:23.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.466+0000 7fd5eca70640 1 Processor -- start 2024-09-18T07:18:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.466+0000 7fd5eca70640 1 -- start start 2024-09-18T07:18:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5eca70640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e8073430 con 0x7fd5e81368e0 2024-09-18T07:18:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5eca70640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e8136cc0 con 0x7fd5e8070b90 2024-09-18T07:18:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5eca70640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e813fec0 con 0x7fd5e8073050 2024-09-18T07:18:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e659c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fd5e8073050 0x7fd5e81361d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48216/0 (socket says 172.21.15.70:48216) 2024-09-18T07:18:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e659c640 1 -- 172.21.15.70:0/3798968629 learned_addr learned my addr 172.21.15.70:0/3798968629 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e6d9d640 1 --1- 172.21.15.70:0/3798968629 >> v1:172.21.15.70:6789/0 conn(0x7fd5e81368e0 0x7fd5e8140af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57630/0 (socket says 172.21.15.70:57630) 2024-09-18T07:18:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e559a640 1 -- 172.21.15.70:0/3798968629 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3875486824 0 0) 0x7fd5e813fec0 con 0x7fd5e8073050 2024-09-18T07:18:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e559a640 1 -- 172.21.15.70:0/3798968629 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5d0004630 con 0x7fd5e8073050 2024-09-18T07:18:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e559a640 1 -- 172.21.15.70:0/3798968629 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3977734113 0 0) 0x7fd5e8073430 con 0x7fd5e81368e0 2024-09-18T07:18:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e559a640 1 -- 172.21.15.70:0/3798968629 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5e813fec0 con 0x7fd5e81368e0 2024-09-18T07:18:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e559a640 1 -- 172.21.15.70:0/3798968629 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3118712608 0 0) 0x7fd5e8136cc0 con 0x7fd5e8070b90 2024-09-18T07:18:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e559a640 1 -- 172.21.15.70:0/3798968629 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5e8073430 con 0x7fd5e8070b90 2024-09-18T07:18:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e559a640 1 -- 172.21.15.70:0/3798968629 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 261547965 0 0) 0x7fd5d0004630 con 0x7fd5e8073050 2024-09-18T07:18:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e559a640 1 -- 172.21.15.70:0/3798968629 >> v1:172.21.15.172:6789/0 conn(0x7fd5e8070b90 legacy=0x7fd5e81403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e559a640 1 -- 172.21.15.70:0/3798968629 >> v1:172.21.15.70:6789/0 conn(0x7fd5e81368e0 legacy=0x7fd5e8140af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e559a640 1 -- 172.21.15.70:0/3798968629 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e8074510 con 0x7fd5e8073050 2024-09-18T07:18:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.467+0000 7fd5e559a640 1 -- 172.21.15.70:0/3798968629 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd5d80015c0 con 0x7fd5e8073050 2024-09-18T07:18:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.468+0000 7fd5e559a640 1 -- 172.21.15.70:0/3798968629 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd5d80017e0 con 0x7fd5e8073050 2024-09-18T07:18:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.468+0000 7fd5e559a640 1 -- 172.21.15.70:0/3798968629 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd5d8001bb0 con 0x7fd5e8073050 2024-09-18T07:18:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.468+0000 7fd5eca70640 1 -- 172.21.15.70:0/3798968629 >> v1:172.21.15.70:6790/0 conn(0x7fd5e8073050 legacy=0x7fd5e81361d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.468+0000 7fd5eca70640 1 -- 172.21.15.70:0/3798968629 shutdown_connections 2024-09-18T07:18:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.468+0000 7fd5eca70640 1 -- 172.21.15.70:0/3798968629 >> 172.21.15.70:0/3798968629 conn(0x7fd5e8076130 msgr2=0x7fd5e8076510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.468+0000 7fd5eca70640 1 -- 172.21.15.70:0/3798968629 shutdown_connections 2024-09-18T07:18:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.468+0000 7fd5eca70640 1 -- 172.21.15.70:0/3798968629 wait complete. 2024-09-18T07:18:23.476 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.470+0000 7fd5eca70640 1 Processor -- start 2024-09-18T07:18:23.476 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.470+0000 7fd5eca70640 1 -- start start 2024-09-18T07:18:23.476 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.470+0000 7fd5eca70640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e813a3d0 con 0x7fd5e8073050 2024-09-18T07:18:23.476 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.470+0000 7fd5eca70640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e81d8ba0 con 0x7fd5e81368e0 2024-09-18T07:18:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.470+0000 7fd5eca70640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e8142290 con 0x7fd5e8070b90 2024-09-18T07:18:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5e659c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fd5e8070b90 0x7fd5e80b0bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48222/0 (socket says 172.21.15.70:48222) 2024-09-18T07:18:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5e659c640 1 -- 172.21.15.70:0/3776254124 learned_addr learned my addr 172.21.15.70:0/3776254124 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5e5d9b640 1 --1- 172.21.15.70:0/3776254124 >> v1:172.21.15.70:6789/0 conn(0x7fd5e8073050 0x7fd5e81d90c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57638/0 (socket says 172.21.15.70:57638) 2024-09-18T07:18:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3564578891 0 0) 0x7fd5e813a3d0 con 0x7fd5e8073050 2024-09-18T07:18:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5c8003520 con 0x7fd5e8073050 2024-09-18T07:18:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3936006179 0 0) 0x7fd5e8142290 con 0x7fd5e8070b90 2024-09-18T07:18:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5e813a3d0 con 0x7fd5e8070b90 2024-09-18T07:18:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1847152781 0 0) 0x7fd5e81d8ba0 con 0x7fd5e81368e0 2024-09-18T07:18:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5e8142290 con 0x7fd5e81368e0 2024-09-18T07:18:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 37684663 0 0) 0x7fd5c8003520 con 0x7fd5e8073050 2024-09-18T07:18:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd5e81d8ba0 con 0x7fd5e8073050 2024-09-18T07:18:23.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd5dc0018f0 con 0x7fd5e8073050 2024-09-18T07:18:23.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1456081033 0 0) 0x7fd5e8142290 con 0x7fd5e81368e0 2024-09-18T07:18:23.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd5c8003520 con 0x7fd5e81368e0 2024-09-18T07:18:23.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd5e0001a30 con 0x7fd5e81368e0 2024-09-18T07:18:23.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1602413998 0 0) 0x7fd5e813a3d0 con 0x7fd5e8070b90 2024-09-18T07:18:23.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd5e8142290 con 0x7fd5e8070b90 2024-09-18T07:18:23.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd5d8001b30 con 0x7fd5e8070b90 2024-09-18T07:18:23.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 93131492 0 0) 0x7fd5e81d8ba0 con 0x7fd5e8073050 2024-09-18T07:18:23.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 >> v1:172.21.15.70:6790/0 conn(0x7fd5e8070b90 legacy=0x7fd5e80b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 >> v1:172.21.15.172:6789/0 conn(0x7fd5e81368e0 legacy=0x7fd5e8132f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e8142460 con 0x7fd5e8073050 2024-09-18T07:18:23.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5eca70640 1 -- 172.21.15.70:0/3776254124 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e81d97d0 con 0x7fd5e8073050 2024-09-18T07:18:23.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5eca70640 1 -- 172.21.15.70:0/3776254124 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd5e80aff80 con 0x7fd5e8073050 2024-09-18T07:18:23.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd5dc003610 con 0x7fd5e8073050 2024-09-18T07:18:23.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.471+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fd5dc003a20 con 0x7fd5e8073050 2024-09-18T07:18:23.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.472+0000 7fd5eca70640 1 -- 172.21.15.70:0/3776254124 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fd5cc000f80 con 0x7fd5e8073050 2024-09-18T07:18:23.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.473+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7fd5dc003a20 con 0x7fd5e8073050 2024-09-18T07:18:23.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.474+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7fd5dc095f80 con 0x7fd5e8073050 2024-09-18T07:18:23.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.474+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 --> v1:172.21.15.70:6804/202753903 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5c8085430 con 0x7fd5c8081860 2024-09-18T07:18:23.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.474+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (unknown 2370098492 0 0) 0x7fd5dc0963e0 con 0x7fd5e8073050 2024-09-18T07:18:23.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.475+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== osd.3 v1:172.21.15.70:6804/202753903 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fd5c8085430 con 0x7fd5c8081860 2024-09-18T07:18:23.494 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.492+0000 7fd5eca70640 1 -- 172.21.15.70:0/3776254124 --> v1:172.21.15.70:6804/202753903 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd5cc002cd0 con 0x7fd5c8081860 2024-09-18T07:18:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.492+0000 7fd5d6ffd640 1 -- 172.21.15.70:0/3776254124 <== osd.3 v1:172.21.15.70:6804/202753903 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7fd5cc002cd0 con 0x7fd5c8081860 2024-09-18T07:18:23.495 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-18T07:18:23.495 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T07:18:23.496 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-18T07:18:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.493+0000 7fd5eca70640 1 -- 172.21.15.70:0/3776254124 >> v1:172.21.15.70:6804/202753903 conn(0x7fd5c8081860 legacy=0x7fd5c8083c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.493+0000 7fd5eca70640 1 -- 172.21.15.70:0/3776254124 >> v1:172.21.15.70:6816/1657990791 conn(0x7fd5c8079a50 legacy=0x7fd5c807bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.493+0000 7fd5eca70640 1 -- 172.21.15.70:0/3776254124 >> v1:172.21.15.70:6789/0 conn(0x7fd5e8073050 legacy=0x7fd5e81d90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.493+0000 7fd5e6d9d640 1 -- 172.21.15.70:0/3776254124 reap_dead start 2024-09-18T07:18:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.493+0000 7fd5eca70640 1 -- 172.21.15.70:0/3776254124 shutdown_connections 2024-09-18T07:18:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.493+0000 7fd5eca70640 1 -- 172.21.15.70:0/3776254124 >> 172.21.15.70:0/3776254124 conn(0x7fd5e8076130 msgr2=0x7fd5e80b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.493+0000 7fd5eca70640 1 -- 172.21.15.70:0/3776254124 shutdown_connections 2024-09-18T07:18:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.493+0000 7fd5eca70640 1 -- 172.21.15.70:0/3776254124 wait complete. 2024-09-18T07:18:23.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-18T07:18:23.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.604+0000 7f6eef1d3640 1 Processor -- start 2024-09-18T07:18:23.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.604+0000 7f6eef1d3640 1 -- start start 2024-09-18T07:18:23.608 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.604+0000 7f6eef1d3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee80b64f0 con 0x7f6ee813a7a0 2024-09-18T07:18:23.608 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.604+0000 7f6eef1d3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee80ab5c0 con 0x7f6ee8073910 2024-09-18T07:18:23.608 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.604+0000 7f6eef1d3640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee80733f0 con 0x7f6ee80b6110 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.604+0000 7f6eed770640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6ee813a7a0 0x7f6ee814bf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57654/0 (socket says 172.21.15.70:57654) 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.604+0000 7f6eecf6f640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f6ee80b6110 0x7f6ee813a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48236/0 (socket says 172.21.15.70:48236) 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.604+0000 7f6eed770640 1 -- 172.21.15.70:0/1009654587 learned_addr learned my addr 172.21.15.70:0/1009654587 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.605+0000 7f6ee77fe640 1 -- 172.21.15.70:0/1009654587 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 763441850 0 0) 0x7f6ee80b64f0 con 0x7f6ee813a7a0 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.605+0000 7f6ee77fe640 1 -- 172.21.15.70:0/1009654587 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ed8003520 con 0x7f6ee813a7a0 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.605+0000 7f6ee77fe640 1 -- 172.21.15.70:0/1009654587 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2169901402 0 0) 0x7f6ee80733f0 con 0x7f6ee80b6110 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.605+0000 7f6ee77fe640 1 -- 172.21.15.70:0/1009654587 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ee80b64f0 con 0x7f6ee80b6110 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.605+0000 7f6ee77fe640 1 -- 172.21.15.70:0/1009654587 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2850473160 0 0) 0x7f6ed8003520 con 0x7f6ee813a7a0 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.605+0000 7f6ee77fe640 1 -- 172.21.15.70:0/1009654587 >> v1:172.21.15.70:6790/0 conn(0x7f6ee80b6110 legacy=0x7f6ee813a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.605+0000 7f6ee77fe640 1 -- 172.21.15.70:0/1009654587 >> v1:172.21.15.172:6789/0 conn(0x7f6ee8073910 legacy=0x7f6ee8143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.605+0000 7f6ee77fe640 1 -- 172.21.15.70:0/1009654587 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee80743d0 con 0x7f6ee813a7a0 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.605+0000 7f6ee77fe640 1 -- 172.21.15.70:0/1009654587 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6ee0001a10 con 0x7f6ee813a7a0 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.605+0000 7f6ee77fe640 1 -- 172.21.15.70:0/1009654587 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6ee0001bb0 con 0x7f6ee813a7a0 2024-09-18T07:18:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.605+0000 7f6ee77fe640 1 -- 172.21.15.70:0/1009654587 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6ee0002040 con 0x7f6ee813a7a0 2024-09-18T07:18:23.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.605+0000 7f6eef1d3640 1 -- 172.21.15.70:0/1009654587 >> v1:172.21.15.70:6789/0 conn(0x7f6ee813a7a0 legacy=0x7f6ee814bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.606+0000 7f6eef1d3640 1 -- 172.21.15.70:0/1009654587 shutdown_connections 2024-09-18T07:18:23.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.606+0000 7f6eef1d3640 1 -- 172.21.15.70:0/1009654587 >> 172.21.15.70:0/1009654587 conn(0x7f6ee8076bb0 msgr2=0x7f6ee80b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:23.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.606+0000 7f6eef1d3640 1 -- 172.21.15.70:0/1009654587 shutdown_connections 2024-09-18T07:18:23.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.606+0000 7f6eef1d3640 1 -- 172.21.15.70:0/1009654587 wait complete. 2024-09-18T07:18:23.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.607+0000 7f6eef1d3640 1 Processor -- start 2024-09-18T07:18:23.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.607+0000 7f6eef1d3640 1 -- start start 2024-09-18T07:18:23.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.607+0000 7f6eef1d3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee81d7510 con 0x7f6ee80b6110 2024-09-18T07:18:23.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.607+0000 7f6eef1d3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee80ac600 con 0x7f6ee813a7a0 2024-09-18T07:18:23.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.607+0000 7f6eef1d3640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee80ac7d0 con 0x7f6ee813b480 2024-09-18T07:18:23.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.607+0000 7f6eecf6f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6ee80b6110 0x7f6ee80abef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57660/0 (socket says 172.21.15.70:57660) 2024-09-18T07:18:23.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.607+0000 7f6eed770640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f6ee813b480 0x7f6ee81e1000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48240/0 (socket says 172.21.15.70:48240) 2024-09-18T07:18:23.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.607+0000 7f6eecf6f640 1 -- 172.21.15.70:0/2012951748 learned_addr learned my addr 172.21.15.70:0/2012951748 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:23.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3408161128 0 0) 0x7f6ee80ac7d0 con 0x7f6ee813b480 2024-09-18T07:18:23.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ecc003520 con 0x7f6ee813b480 2024-09-18T07:18:23.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1678560454 0 0) 0x7f6ee81d7510 con 0x7f6ee80b6110 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ee80ac7d0 con 0x7f6ee80b6110 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2989448524 0 0) 0x7f6ee80ac600 con 0x7f6ee813a7a0 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ee81d7510 con 0x7f6ee813a7a0 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1392606599 0 0) 0x7f6ecc003520 con 0x7f6ee813b480 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6ee80ac600 con 0x7f6ee813b480 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6ee0001f00 con 0x7f6ee813b480 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3920992183 0 0) 0x7f6ee80ac7d0 con 0x7f6ee80b6110 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6ecc003520 con 0x7f6ee80b6110 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6edc0013b0 con 0x7f6ee80b6110 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2856474145 0 0) 0x7f6ee81d7510 con 0x7f6ee813a7a0 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6ee80ac7d0 con 0x7f6ee813a7a0 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6ed4001830 con 0x7f6ee813a7a0 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1096987759 0 0) 0x7f6ee80ac600 con 0x7f6ee813b480 2024-09-18T07:18:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 >> v1:172.21.15.172:6789/0 conn(0x7f6ee813a7a0 legacy=0x7f6ee81d9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 >> v1:172.21.15.70:6789/0 conn(0x7f6ee80b6110 legacy=0x7f6ee80abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee81da370 con 0x7f6ee813b480 2024-09-18T07:18:23.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6eef1d3640 1 -- 172.21.15.70:0/2012951748 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee81da660 con 0x7f6ee813b480 2024-09-18T07:18:23.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6eef1d3640 1 -- 172.21.15.70:0/2012951748 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f6ee81e2900 con 0x7f6ee813b480 2024-09-18T07:18:23.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6ee0001b10 con 0x7f6ee813b480 2024-09-18T07:18:23.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.608+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f6ee0004fb0 con 0x7f6ee813b480 2024-09-18T07:18:23.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.609+0000 7f6eef1d3640 1 -- 172.21.15.70:0/2012951748 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec8005080 con 0x7f6ee813b480 2024-09-18T07:18:23.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.610+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f6ee001dbe0 con 0x7f6ee813b480 2024-09-18T07:18:23.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.610+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f6ee0096140 con 0x7f6ee813b480 2024-09-18T07:18:23.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.614+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f6ee005e9a0 con 0x7f6ee813b480 2024-09-18T07:18:23.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.789+0000 7f6eef1d3640 1 -- 172.21.15.70:0/2012951748 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f6ec8000b90 con 0x7f6ecc079a00 2024-09-18T07:18:23.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.789+0000 7f6ee57fa640 1 -- 172.21.15.70:0/2012951748 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (unknown 3526570807 0 0) 0x7f6ec8000b90 con 0x7f6ecc079a00 2024-09-18T07:18:23.792 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 3 to deep-scrub 2024-09-18T07:18:23.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.791+0000 7f6eef1d3640 1 -- 172.21.15.70:0/2012951748 >> v1:172.21.15.70:6816/1657990791 conn(0x7f6ecc079a00 legacy=0x7f6ecc07bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.791+0000 7f6eef1d3640 1 -- 172.21.15.70:0/2012951748 >> v1:172.21.15.70:6790/0 conn(0x7f6ee813b480 legacy=0x7f6ee81e1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.791+0000 7f6eef1d3640 1 -- 172.21.15.70:0/2012951748 shutdown_connections 2024-09-18T07:18:23.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.791+0000 7f6eef1d3640 1 -- 172.21.15.70:0/2012951748 >> 172.21.15.70:0/2012951748 conn(0x7f6ee8076bb0 msgr2=0x7f6ee81405b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:23.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.791+0000 7f6eef1d3640 1 -- 172.21.15.70:0/2012951748 shutdown_connections 2024-09-18T07:18:23.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.791+0000 7f6eef1d3640 1 -- 172.21.15.70:0/2012951748 wait complete. 2024-09-18T07:18:23.804 INFO:tasks.ceph:Scrubbing osd.4 2024-09-18T07:18:23.804 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-18T07:18:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.903+0000 7feab45ca640 1 Processor -- start 2024-09-18T07:18:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.903+0000 7feab45ca640 1 -- start start 2024-09-18T07:18:23.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.904+0000 7feab45ca640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaac068fc0 con 0x7feaac0b2670 2024-09-18T07:18:23.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.904+0000 7feab45ca640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaac072aa0 con 0x7feaac140eb0 2024-09-18T07:18:23.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.904+0000 7feab45ca640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaac074b20 con 0x7feaac068be0 2024-09-18T07:18:23.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.904+0000 7feab2366640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7feaac068be0 0x7feaac1407a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48256/0 (socket says 172.21.15.70:48256) 2024-09-18T07:18:23.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.904+0000 7feab2366640 1 -- 172.21.15.70:0/1071303351 learned_addr learned my addr 172.21.15.70:0/1071303351 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:23.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.904+0000 7feab1b65640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7feaac0b2670 0x7feaac146df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57674/0 (socket says 172.21.15.70:57674) 2024-09-18T07:18:23.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.904+0000 7feab1364640 1 -- 172.21.15.70:0/1071303351 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 112286251 0 0) 0x7feaac068fc0 con 0x7feaac0b2670 2024-09-18T07:18:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.904+0000 7feab1364640 1 -- 172.21.15.70:0/1071303351 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea98003520 con 0x7feaac0b2670 2024-09-18T07:18:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.904+0000 7feab1364640 1 -- 172.21.15.70:0/1071303351 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2299292585 0 0) 0x7feaac074b20 con 0x7feaac068be0 2024-09-18T07:18:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.904+0000 7feab1364640 1 -- 172.21.15.70:0/1071303351 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feaac068fc0 con 0x7feaac068be0 2024-09-18T07:18:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.905+0000 7feab1364640 1 -- 172.21.15.70:0/1071303351 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3011893609 0 0) 0x7fea98003520 con 0x7feaac0b2670 2024-09-18T07:18:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.905+0000 7feab1364640 1 -- 172.21.15.70:0/1071303351 >> v1:172.21.15.70:6790/0 conn(0x7feaac068be0 legacy=0x7feaac1407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.905+0000 7feab1364640 1 -- 172.21.15.70:0/1071303351 >> v1:172.21.15.172:6789/0 conn(0x7feaac140eb0 legacy=0x7feaac147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.905+0000 7feab1364640 1 -- 172.21.15.70:0/1071303351 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac0ae8f0 con 0x7feaac0b2670 2024-09-18T07:18:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.905+0000 7feab1364640 1 -- 172.21.15.70:0/1071303351 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fea9c001690 con 0x7feaac0b2670 2024-09-18T07:18:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.905+0000 7feab1364640 1 -- 172.21.15.70:0/1071303351 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fea9c0018b0 con 0x7feaac0b2670 2024-09-18T07:18:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.905+0000 7feab1364640 1 -- 172.21.15.70:0/1071303351 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7fea9c001cc0 con 0x7feaac0b2670 2024-09-18T07:18:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.905+0000 7feab45ca640 1 -- 172.21.15.70:0/1071303351 >> v1:172.21.15.70:6789/0 conn(0x7feaac0b2670 legacy=0x7feaac146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.905+0000 7feab45ca640 1 -- 172.21.15.70:0/1071303351 shutdown_connections 2024-09-18T07:18:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.905+0000 7feab45ca640 1 -- 172.21.15.70:0/1071303351 >> 172.21.15.70:0/1071303351 conn(0x7feaac0b6090 msgr2=0x7feaac0b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.905+0000 7feab45ca640 1 -- 172.21.15.70:0/1071303351 shutdown_connections 2024-09-18T07:18:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.905+0000 7feab45ca640 1 -- 172.21.15.70:0/1071303351 wait complete. 2024-09-18T07:18:23.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.906+0000 7feab45ca640 1 Processor -- start 2024-09-18T07:18:23.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.906+0000 7feab45ca640 1 -- start start 2024-09-18T07:18:23.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feab45ca640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaac148b20 con 0x7feaac140eb0 2024-09-18T07:18:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feab45ca640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaac13d930 con 0x7feaac0b2670 2024-09-18T07:18:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feab45ca640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaac13db00 con 0x7feaac068be0 2024-09-18T07:18:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feab2366640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7feaac068be0 0x7feaac142ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48266/0 (socket says 172.21.15.70:48266) 2024-09-18T07:18:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feab2366640 1 -- 172.21.15.70:0/1172667874 learned_addr learned my addr 172.21.15.70:0/1172667874 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625287534 0 0) 0x7feaac13db00 con 0x7feaac068be0 2024-09-18T07:18:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea90003520 con 0x7feaac068be0 2024-09-18T07:18:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3302439414 0 0) 0x7feaac148b20 con 0x7feaac140eb0 2024-09-18T07:18:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feaac13db00 con 0x7feaac140eb0 2024-09-18T07:18:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2127664614 0 0) 0x7fea90003520 con 0x7feaac068be0 2024-09-18T07:18:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7feaac148b20 con 0x7feaac068be0 2024-09-18T07:18:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2725862765 0 0) 0x7feaac13d930 con 0x7feaac0b2670 2024-09-18T07:18:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea90003520 con 0x7feaac0b2670 2024-09-18T07:18:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2845615347 0 0) 0x7feaac13db00 con 0x7feaac140eb0 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7feaac13d930 con 0x7feaac140eb0 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7feaa80012d0 con 0x7feaac068be0 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.907+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7feaa4001a90 con 0x7feaac140eb0 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.908+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1672469425 0 0) 0x7feaac148b20 con 0x7feaac068be0 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.908+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 >> v1:172.21.15.172:6789/0 conn(0x7feaac0b2670 legacy=0x7feaac1431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.908+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 >> v1:172.21.15.70:6789/0 conn(0x7feaac140eb0 legacy=0x7feaac1e2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.908+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac1dbda0 con 0x7feaac068be0 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.908+0000 7feab45ca640 1 -- 172.21.15.70:0/1172667874 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7feaac1dc030 con 0x7feaac068be0 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.908+0000 7feab45ca640 1 -- 172.21.15.70:0/1172667874 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7feaac0b1590 con 0x7feaac068be0 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.908+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7feaa8001610 con 0x7feaac068be0 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.908+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7feaa8004c30 con 0x7feaac068be0 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.909+0000 7feab45ca640 1 -- 172.21.15.70:0/1172667874 --> v1:172.21.15.70:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fea94000f80 con 0x7feaac068be0 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.909+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7feaa801da00 con 0x7feaac068be0 2024-09-18T07:18:23.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.910+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7feaa8096240 con 0x7feaac068be0 2024-09-18T07:18:23.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.910+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 --> v1:172.21.15.172:6808/3405680289 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea90085260 con 0x7fea90081690 2024-09-18T07:18:23.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.910+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (unknown 2370098492 0 0) 0x7feaa8096650 con 0x7feaac068be0 2024-09-18T07:18:23.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.911+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== osd.4 v1:172.21.15.172:6808/3405680289 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fea90085260 con 0x7fea90081690 2024-09-18T07:18:23.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.923+0000 7feab45ca640 1 -- 172.21.15.70:0/1172667874 --> v1:172.21.15.172:6808/3405680289 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fea94002cd0 con 0x7fea90081690 2024-09-18T07:18:23.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.923+0000 7feaa2ffd640 1 -- 172.21.15.70:0/1172667874 <== osd.4 v1:172.21.15.172:6808/3405680289 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7fea94002cd0 con 0x7fea90081690 2024-09-18T07:18:23.926 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-18T07:18:23.926 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T07:18:23.926 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-18T07:18:23.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.924+0000 7feab45ca640 1 -- 172.21.15.70:0/1172667874 >> v1:172.21.15.172:6808/3405680289 conn(0x7fea90081690 legacy=0x7fea90083ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.924+0000 7feab45ca640 1 -- 172.21.15.70:0/1172667874 >> v1:172.21.15.70:6816/1657990791 conn(0x7fea90079880 legacy=0x7fea9007bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.924+0000 7feab45ca640 1 -- 172.21.15.70:0/1172667874 >> v1:172.21.15.70:6790/0 conn(0x7feaac068be0 legacy=0x7feaac142ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:23.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.924+0000 7feab2b67640 1 -- 172.21.15.70:0/1172667874 reap_dead start 2024-09-18T07:18:23.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.924+0000 7feab45ca640 1 -- 172.21.15.70:0/1172667874 shutdown_connections 2024-09-18T07:18:23.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.924+0000 7feab45ca640 1 -- 172.21.15.70:0/1172667874 >> 172.21.15.70:0/1172667874 conn(0x7feaac0b6090 msgr2=0x7feaac13f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:23.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.924+0000 7feab45ca640 1 -- 172.21.15.70:0/1172667874 shutdown_connections 2024-09-18T07:18:23.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:23.924+0000 7feab45ca640 1 -- 172.21.15.70:0/1172667874 wait complete. 2024-09-18T07:18:23.937 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-18T07:18:24.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.032+0000 7f4d63f65640 1 Processor -- start 2024-09-18T07:18:24.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.032+0000 7f4d63f65640 1 -- start start 2024-09-18T07:18:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d63f65640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d5c0aaaa0 con 0x7f4d5c072d70 2024-09-18T07:18:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d63f65640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d5c0aad80 con 0x7f4d5c140980 2024-09-18T07:18:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d63f65640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d5c074410 con 0x7f4d5c072990 2024-09-18T07:18:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d61d01640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f4d5c072990 0x7f4d5c0b5ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48268/0 (socket says 172.21.15.70:48268) 2024-09-18T07:18:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d61d01640 1 -- 172.21.15.70:0/1653146811 learned_addr learned my addr 172.21.15.70:0/1653146811 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:24.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d61500640 1 --1- 172.21.15.70:0/1653146811 >> v1:172.21.15.70:6789/0 conn(0x7f4d5c072d70 0x7f4d5c140270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57696/0 (socket says 172.21.15.70:57696) 2024-09-18T07:18:24.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d60cff640 1 -- 172.21.15.70:0/1653146811 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 278862483 0 0) 0x7f4d5c0aaaa0 con 0x7f4d5c072d70 2024-09-18T07:18:24.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d60cff640 1 -- 172.21.15.70:0/1653146811 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d44003520 con 0x7f4d5c072d70 2024-09-18T07:18:24.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d60cff640 1 -- 172.21.15.70:0/1653146811 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2961929210 0 0) 0x7f4d5c074410 con 0x7f4d5c072990 2024-09-18T07:18:24.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d60cff640 1 -- 172.21.15.70:0/1653146811 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d5c0aaaa0 con 0x7f4d5c072990 2024-09-18T07:18:24.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d60cff640 1 -- 172.21.15.70:0/1653146811 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3539179672 0 0) 0x7f4d44003520 con 0x7f4d5c072d70 2024-09-18T07:18:24.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d60cff640 1 -- 172.21.15.70:0/1653146811 >> v1:172.21.15.70:6790/0 conn(0x7f4d5c072990 legacy=0x7f4d5c0b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d60cff640 1 -- 172.21.15.70:0/1653146811 >> v1:172.21.15.172:6789/0 conn(0x7f4d5c140980 legacy=0x7f4d5c14ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d60cff640 1 -- 172.21.15.70:0/1653146811 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5c073e50 con 0x7f4d5c072d70 2024-09-18T07:18:24.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.033+0000 7f4d60cff640 1 -- 172.21.15.70:0/1653146811 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4d50001a10 con 0x7f4d5c072d70 2024-09-18T07:18:24.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.034+0000 7f4d60cff640 1 -- 172.21.15.70:0/1653146811 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4d50001c30 con 0x7f4d5c072d70 2024-09-18T07:18:24.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.034+0000 7f4d60cff640 1 -- 172.21.15.70:0/1653146811 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4d500020c0 con 0x7f4d5c072d70 2024-09-18T07:18:24.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.034+0000 7f4d63f65640 1 -- 172.21.15.70:0/1653146811 >> v1:172.21.15.70:6789/0 conn(0x7f4d5c072d70 legacy=0x7f4d5c140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.034+0000 7f4d63f65640 1 -- 172.21.15.70:0/1653146811 shutdown_connections 2024-09-18T07:18:24.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.034+0000 7f4d63f65640 1 -- 172.21.15.70:0/1653146811 >> 172.21.15.70:0/1653146811 conn(0x7f4d5c076bb0 msgr2=0x7f4d5c0b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:24.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.034+0000 7f4d63f65640 1 -- 172.21.15.70:0/1653146811 shutdown_connections 2024-09-18T07:18:24.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.034+0000 7f4d63f65640 1 -- 172.21.15.70:0/1653146811 wait complete. 2024-09-18T07:18:24.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.035+0000 7f4d63f65640 1 Processor -- start 2024-09-18T07:18:24.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.035+0000 7f4d63f65640 1 -- start start 2024-09-18T07:18:24.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d63f65640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d5c0abe60 con 0x7f4d5c072d70 2024-09-18T07:18:24.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d63f65640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d5c1396c0 con 0x7f4d5c072990 2024-09-18T07:18:24.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d63f65640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d5c1d51d0 con 0x7f4d5c140980 2024-09-18T07:18:24.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d61500640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4d5c072d70 0x7f4d5c138970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57698/0 (socket says 172.21.15.70:57698) 2024-09-18T07:18:24.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d62502640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f4d5c140980 0x7f4d5c1429f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48280/0 (socket says 172.21.15.70:48280) 2024-09-18T07:18:24.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d61500640 1 -- 172.21.15.70:0/489013738 learned_addr learned my addr 172.21.15.70:0/489013738 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2851860642 0 0) 0x7f4d5c0abe60 con 0x7f4d5c072d70 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d48003520 con 0x7f4d5c072d70 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 930378238 0 0) 0x7f4d5c1d51d0 con 0x7f4d5c140980 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d5c0abe60 con 0x7f4d5c140980 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3092928802 0 0) 0x7f4d5c1396c0 con 0x7f4d5c072990 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d5c1d51d0 con 0x7f4d5c072990 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3480599351 0 0) 0x7f4d48003520 con 0x7f4d5c072d70 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4d5c1396c0 con 0x7f4d5c072d70 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4d50001f80 con 0x7f4d5c072d70 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1336546046 0 0) 0x7f4d5c0abe60 con 0x7f4d5c140980 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4d48003520 con 0x7f4d5c140980 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4d54001750 con 0x7f4d5c140980 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 757448974 0 0) 0x7f4d5c1d51d0 con 0x7f4d5c072990 2024-09-18T07:18:24.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4d5c0abe60 con 0x7f4d5c072990 2024-09-18T07:18:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4d4c001210 con 0x7f4d5c072990 2024-09-18T07:18:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 106548653 0 0) 0x7f4d5c1396c0 con 0x7f4d5c072d70 2024-09-18T07:18:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.036+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 >> v1:172.21.15.70:6790/0 conn(0x7f4d5c140980 legacy=0x7f4d5c1429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.037+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 >> v1:172.21.15.172:6789/0 conn(0x7f4d5c072990 legacy=0x7f4d5c0acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.037+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5c0ac360 con 0x7f4d5c072d70 2024-09-18T07:18:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.037+0000 7f4d63f65640 1 -- 172.21.15.70:0/489013738 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5c0ac5f0 con 0x7f4d5c072d70 2024-09-18T07:18:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.037+0000 7f4d63f65640 1 -- 172.21.15.70:0/489013738 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4d5c139260 con 0x7f4d5c072d70 2024-09-18T07:18:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.037+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4d50004f30 con 0x7f4d5c072d70 2024-09-18T07:18:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.037+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f4d50005280 con 0x7f4d5c072d70 2024-09-18T07:18:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.038+0000 7f4d63f65640 1 -- 172.21.15.70:0/489013738 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d3c005080 con 0x7f4d5c072d70 2024-09-18T07:18:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.038+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f4d5001e0d0 con 0x7f4d5c072d70 2024-09-18T07:18:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.039+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f4d50096430 con 0x7f4d5c072d70 2024-09-18T07:18:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.043+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f4d50001c00 con 0x7f4d5c072d70 2024-09-18T07:18:24.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.215+0000 7f4d63f65640 1 -- 172.21.15.70:0/489013738 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f4d3c000b90 con 0x7f4d48079980 2024-09-18T07:18:24.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.216+0000 7f4d5a7fc640 1 -- 172.21.15.70:0/489013738 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (unknown 3782719169 0 0) 0x7f4d3c000b90 con 0x7f4d48079980 2024-09-18T07:18:24.220 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 4 to deep-scrub 2024-09-18T07:18:24.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.218+0000 7f4d63f65640 1 -- 172.21.15.70:0/489013738 >> v1:172.21.15.70:6816/1657990791 conn(0x7f4d48079980 legacy=0x7f4d4807be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.218+0000 7f4d63f65640 1 -- 172.21.15.70:0/489013738 >> v1:172.21.15.70:6789/0 conn(0x7f4d5c072d70 legacy=0x7f4d5c138970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.218+0000 7f4d63f65640 1 -- 172.21.15.70:0/489013738 shutdown_connections 2024-09-18T07:18:24.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.218+0000 7f4d63f65640 1 -- 172.21.15.70:0/489013738 >> 172.21.15.70:0/489013738 conn(0x7f4d5c076bb0 msgr2=0x7f4d5c13c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:24.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.218+0000 7f4d63f65640 1 -- 172.21.15.70:0/489013738 shutdown_connections 2024-09-18T07:18:24.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.218+0000 7f4d63f65640 1 -- 172.21.15.70:0/489013738 wait complete. 2024-09-18T07:18:24.232 INFO:tasks.ceph:Scrubbing osd.5 2024-09-18T07:18:24.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-18T07:18:24.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.339+0000 7f25b0c33640 1 Processor -- start 2024-09-18T07:18:24.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.339+0000 7f25b0c33640 1 -- start start 2024-09-18T07:18:24.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.339+0000 7f25b0c33640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ac0b5850 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.339+0000 7f25b0c33640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ac073400 con 0x7f25ac143470 2024-09-18T07:18:24.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25b0c33640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ac072260 con 0x7f25ac0b5470 2024-09-18T07:18:24.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25aa59c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f25ac0b5470 0x7f25ac142d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48284/0 (socket says 172.21.15.70:48284) 2024-09-18T07:18:24.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25aa59c640 1 -- 172.21.15.70:0/1484179302 learned_addr learned my addr 172.21.15.70:0/1484179302 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25a959a640 1 -- 172.21.15.70:0/1484179302 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1001002529 0 0) 0x7f25ac0b5850 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25a959a640 1 -- 172.21.15.70:0/1484179302 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2594003520 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25a959a640 1 -- 172.21.15.70:0/1484179302 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1143635912 0 0) 0x7f25ac072260 con 0x7f25ac0b5470 2024-09-18T07:18:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25a959a640 1 -- 172.21.15.70:0/1484179302 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25ac0b5850 con 0x7f25ac0b5470 2024-09-18T07:18:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25a959a640 1 -- 172.21.15.70:0/1484179302 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3149864052 0 0) 0x7f25ac073400 con 0x7f25ac143470 2024-09-18T07:18:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25a959a640 1 -- 172.21.15.70:0/1484179302 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25ac072260 con 0x7f25ac143470 2024-09-18T07:18:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25a959a640 1 -- 172.21.15.70:0/1484179302 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3777724239 0 0) 0x7f25ac0b5850 con 0x7f25ac0b5470 2024-09-18T07:18:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25a959a640 1 -- 172.21.15.70:0/1484179302 >> v1:172.21.15.172:6789/0 conn(0x7f25ac143470 legacy=0x7f25ac144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25a959a640 1 -- 172.21.15.70:0/1484179302 >> v1:172.21.15.70:6789/0 conn(0x7f25ac0b3bd0 legacy=0x7f25ac143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25a959a640 1 -- 172.21.15.70:0/1484179302 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac144b80 con 0x7f25ac0b5470 2024-09-18T07:18:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25a959a640 1 -- 172.21.15.70:0/1484179302 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f259c001670 con 0x7f25ac0b5470 2024-09-18T07:18:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25a959a640 1 -- 172.21.15.70:0/1484179302 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f259c001810 con 0x7f25ac0b5470 2024-09-18T07:18:24.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.340+0000 7f25a959a640 1 -- 172.21.15.70:0/1484179302 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f259c001ca0 con 0x7f25ac0b5470 2024-09-18T07:18:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.341+0000 7f25b0c33640 1 -- 172.21.15.70:0/1484179302 >> v1:172.21.15.70:6790/0 conn(0x7f25ac0b5470 legacy=0x7f25ac142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.341+0000 7f25b0c33640 1 -- 172.21.15.70:0/1484179302 shutdown_connections 2024-09-18T07:18:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.341+0000 7f25b0c33640 1 -- 172.21.15.70:0/1484179302 >> 172.21.15.70:0/1484179302 conn(0x7f25ac0b62f0 msgr2=0x7f25ac0b66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.341+0000 7f25b0c33640 1 -- 172.21.15.70:0/1484179302 shutdown_connections 2024-09-18T07:18:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.341+0000 7f25b0c33640 1 -- 172.21.15.70:0/1484179302 wait complete. 2024-09-18T07:18:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.342+0000 7f25b0c33640 1 Processor -- start 2024-09-18T07:18:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.342+0000 7f25b0c33640 1 -- start start 2024-09-18T07:18:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.343+0000 7f25b0c33640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ac146a50 con 0x7f25ac0b5470 2024-09-18T07:18:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.343+0000 7f25b0c33640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ac1dc980 con 0x7f25ac143470 2024-09-18T07:18:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.343+0000 7f25b0c33640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ac1dcb50 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.343+0000 7f25aa59c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f25ac0b3bd0 0x7f25ac1dc270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48298/0 (socket says 172.21.15.70:48298) 2024-09-18T07:18:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.343+0000 7f25aa59c640 1 -- 172.21.15.70:0/2371756139 learned_addr learned my addr 172.21.15.70:0/2371756139 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.343+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1853826436 0 0) 0x7f25ac1dcb50 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.343+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f258c003520 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.343+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 644638472 0 0) 0x7f25ac146a50 con 0x7f25ac0b5470 2024-09-18T07:18:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.343+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25ac1dcb50 con 0x7f25ac0b5470 2024-09-18T07:18:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.343+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1670805081 0 0) 0x7f25ac1dc980 con 0x7f25ac143470 2024-09-18T07:18:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.343+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25ac146a50 con 0x7f25ac143470 2024-09-18T07:18:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2009641471 0 0) 0x7f258c003520 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f25ac1dc980 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f259c001c20 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2379684148 0 0) 0x7f25ac1dcb50 con 0x7f25ac0b5470 2024-09-18T07:18:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f258c003520 con 0x7f25ac0b5470 2024-09-18T07:18:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f25a0001150 con 0x7f25ac0b5470 2024-09-18T07:18:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 685166019 0 0) 0x7f25ac1dc980 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 >> v1:172.21.15.172:6789/0 conn(0x7f25ac143470 legacy=0x7f25ac1e30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 >> v1:172.21.15.70:6789/0 conn(0x7f25ac0b5470 legacy=0x7f25ac136a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac139850 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f25b0c33640 1 -- 172.21.15.70:0/2371756139 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f25ac139b60 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f25b0c33640 1 -- 172.21.15.70:0/2371756139 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f25ac137360 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f259c004c60 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.344+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f259c005030 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.345+0000 7f25b0c33640 1 -- 172.21.15.70:0/2371756139 --> v1:172.21.15.70:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f2590000f80 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.346+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f259c01de00 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.346+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f259c096200 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.346+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 --> v1:172.21.15.172:6804/3053548626 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f259c001810 con 0x7f258c0817e0 2024-09-18T07:18:24.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.346+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (unknown 2370098492 0 0) 0x7f259c096670 con 0x7f25ac0b3bd0 2024-09-18T07:18:24.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.347+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== osd.5 v1:172.21.15.172:6804/3053548626 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f259c001810 con 0x7f258c0817e0 2024-09-18T07:18:24.367 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.365+0000 7f25b0c33640 1 -- 172.21.15.70:0/2371756139 --> v1:172.21.15.172:6804/3053548626 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2590002cd0 con 0x7f258c0817e0 2024-09-18T07:18:24.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.365+0000 7f259affd640 1 -- 172.21.15.70:0/2371756139 <== osd.5 v1:172.21.15.172:6804/3053548626 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f2590002cd0 con 0x7f258c0817e0 2024-09-18T07:18:24.368 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-18T07:18:24.368 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T07:18:24.368 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-18T07:18:24.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.365+0000 7f25b0c33640 1 -- 172.21.15.70:0/2371756139 >> v1:172.21.15.172:6804/3053548626 conn(0x7f258c0817e0 legacy=0x7f258c083c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.366+0000 7f25b0c33640 1 -- 172.21.15.70:0/2371756139 >> v1:172.21.15.70:6816/1657990791 conn(0x7f258c0799f0 legacy=0x7f258c07beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.366+0000 7f25b0c33640 1 -- 172.21.15.70:0/2371756139 >> v1:172.21.15.70:6790/0 conn(0x7f25ac0b3bd0 legacy=0x7f25ac1dc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.366+0000 7f25aad9d640 1 -- 172.21.15.70:0/2371756139 reap_dead start 2024-09-18T07:18:24.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.366+0000 7f25b0c33640 1 -- 172.21.15.70:0/2371756139 shutdown_connections 2024-09-18T07:18:24.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.366+0000 7f25b0c33640 1 -- 172.21.15.70:0/2371756139 >> 172.21.15.70:0/2371756139 conn(0x7f25ac0b62f0 msgr2=0x7f25ac070f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:24.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.366+0000 7f25b0c33640 1 -- 172.21.15.70:0/2371756139 shutdown_connections 2024-09-18T07:18:24.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.366+0000 7f25b0c33640 1 -- 172.21.15.70:0/2371756139 wait complete. 2024-09-18T07:18:24.379 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-18T07:18:24.498 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.496+0000 7f9c728dd640 1 Processor -- start 2024-09-18T07:18:24.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.496+0000 7f9c728dd640 1 -- start start 2024-09-18T07:18:24.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.497+0000 7f9c728dd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c6c073680 con 0x7f9c6c13a5f0 2024-09-18T07:18:24.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.497+0000 7f9c728dd640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c6c071670 con 0x7f9c6c072ec0 2024-09-18T07:18:24.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.497+0000 7f9c728dd640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c6c072990 con 0x7f9c6c0732a0 2024-09-18T07:18:24.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.497+0000 7f9c70e7a640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f9c6c0732a0 0x7f9c6c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48302/0 (socket says 172.21.15.70:48302) 2024-09-18T07:18:24.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.497+0000 7f9c70e7a640 1 -- 172.21.15.70:0/1260393945 learned_addr learned my addr 172.21.15.70:0/1260393945 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:24.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.497+0000 7f9c6b7fe640 1 --1- 172.21.15.70:0/1260393945 >> v1:172.21.15.70:6789/0 conn(0x7f9c6c13a5f0 0x7f9c6c071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57714/0 (socket says 172.21.15.70:57714) 2024-09-18T07:18:24.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.497+0000 7f9c6affd640 1 -- 172.21.15.70:0/1260393945 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1843965592 0 0) 0x7f9c60001110 con 0x7f9c6c13a5f0 2024-09-18T07:18:24.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.497+0000 7f9c6affd640 1 -- 172.21.15.70:0/1260393945 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c54003520 con 0x7f9c6c13a5f0 2024-09-18T07:18:24.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.497+0000 7f9c6affd640 1 -- 172.21.15.70:0/1260393945 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2590602913 0 0) 0x7f9c6c072990 con 0x7f9c6c0732a0 2024-09-18T07:18:24.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.497+0000 7f9c6affd640 1 -- 172.21.15.70:0/1260393945 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c60001110 con 0x7f9c6c0732a0 2024-09-18T07:18:24.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.497+0000 7f9c6affd640 1 -- 172.21.15.70:0/1260393945 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2947931288 0 0) 0x7f9c6c071670 con 0x7f9c6c072ec0 2024-09-18T07:18:24.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.497+0000 7f9c6affd640 1 -- 172.21.15.70:0/1260393945 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c6c072990 con 0x7f9c6c072ec0 2024-09-18T07:18:24.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.497+0000 7f9c6affd640 1 -- 172.21.15.70:0/1260393945 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3183622109 0 0) 0x7f9c60001110 con 0x7f9c6c0732a0 2024-09-18T07:18:24.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.498+0000 7f9c6affd640 1 -- 172.21.15.70:0/1260393945 >> v1:172.21.15.172:6789/0 conn(0x7f9c6c072ec0 legacy=0x7f9c6c139ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.498+0000 7f9c6affd640 1 -- 172.21.15.70:0/1260393945 >> v1:172.21.15.70:6789/0 conn(0x7f9c6c13a5f0 legacy=0x7f9c6c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.498+0000 7f9c6affd640 1 -- 172.21.15.70:0/1260393945 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c144bb0 con 0x7f9c6c0732a0 2024-09-18T07:18:24.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.498+0000 7f9c6affd640 1 -- 172.21.15.70:0/1260393945 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f9c640019f0 con 0x7f9c6c0732a0 2024-09-18T07:18:24.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.498+0000 7f9c6affd640 1 -- 172.21.15.70:0/1260393945 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9c64001b90 con 0x7f9c6c0732a0 2024-09-18T07:18:24.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.498+0000 7f9c6affd640 1 -- 172.21.15.70:0/1260393945 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f9c64001fa0 con 0x7f9c6c0732a0 2024-09-18T07:18:24.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.498+0000 7f9c728dd640 1 -- 172.21.15.70:0/1260393945 >> v1:172.21.15.70:6790/0 conn(0x7f9c6c0732a0 legacy=0x7f9c6c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.503 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.498+0000 7f9c728dd640 1 -- 172.21.15.70:0/1260393945 shutdown_connections 2024-09-18T07:18:24.503 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.498+0000 7f9c728dd640 1 -- 172.21.15.70:0/1260393945 >> 172.21.15.70:0/1260393945 conn(0x7f9c6c0b64d0 msgr2=0x7f9c6c0b68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:24.503 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.498+0000 7f9c728dd640 1 -- 172.21.15.70:0/1260393945 shutdown_connections 2024-09-18T07:18:24.503 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.498+0000 7f9c728dd640 1 -- 172.21.15.70:0/1260393945 wait complete. 2024-09-18T07:18:24.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.499+0000 7f9c728dd640 1 Processor -- start 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.500+0000 7f9c728dd640 1 -- start start 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.500+0000 7f9c6bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9c6c072ec0 0x7f9c6c133320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57730/0 (socket says 172.21.15.70:57730) 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.500+0000 7f9c6bfff640 1 -- 172.21.15.70:0/1369188390 learned_addr learned my addr 172.21.15.70:0/1369188390 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c728dd640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c6c1476a0 con 0x7f9c6c072ec0 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 741000286 0 0) 0x7f9c6c072990 con 0x7f9c6c072ec0 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c728dd640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c6c147980 con 0x7f9c6c13a5f0 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c728dd640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c6c147c60 con 0x7f9c6c0732a0 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c58003520 con 0x7f9c6c072ec0 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2543894494 0 0) 0x7f9c6c147c60 con 0x7f9c6c0732a0 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c6c072990 con 0x7f9c6c0732a0 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 651230449 0 0) 0x7f9c6c147980 con 0x7f9c6c13a5f0 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c6c147c60 con 0x7f9c6c13a5f0 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4016084737 0 0) 0x7f9c58003520 con 0x7f9c6c072ec0 2024-09-18T07:18:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9c6c147980 con 0x7f9c6c072ec0 2024-09-18T07:18:24.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f9c5c001a10 con 0x7f9c6c072ec0 2024-09-18T07:18:24.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 30690402 0 0) 0x7f9c6c072990 con 0x7f9c6c0732a0 2024-09-18T07:18:24.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9c58003520 con 0x7f9c6c0732a0 2024-09-18T07:18:24.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1804834925 0 0) 0x7f9c6c147c60 con 0x7f9c6c13a5f0 2024-09-18T07:18:24.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.501+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9c6c072990 con 0x7f9c6c13a5f0 2024-09-18T07:18:24.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.502+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1874073926 0 0) 0x7f9c6c147980 con 0x7f9c6c072ec0 2024-09-18T07:18:24.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.502+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 >> v1:172.21.15.70:6790/0 conn(0x7f9c6c0732a0 legacy=0x7f9c6c146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.502+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 >> v1:172.21.15.172:6789/0 conn(0x7f9c6c13a5f0 legacy=0x7f9c6c13d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.508 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.502+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c12d890 con 0x7f9c6c072ec0 2024-09-18T07:18:24.508 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.502+0000 7f9c728dd640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6c145770 con 0x7f9c6c072ec0 2024-09-18T07:18:24.508 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.502+0000 7f9c728dd640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9c6c1e47c0 con 0x7f9c6c072ec0 2024-09-18T07:18:24.508 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.502+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9c5c001cb0 con 0x7f9c6c072ec0 2024-09-18T07:18:24.508 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.503+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f9c5c003a60 con 0x7f9c6c072ec0 2024-09-18T07:18:24.508 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.503+0000 7f9c728dd640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c4c005080 con 0x7f9c6c072ec0 2024-09-18T07:18:24.509 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.504+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f9c5c003a60 con 0x7f9c6c072ec0 2024-09-18T07:18:24.509 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.504+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f9c5c095f20 con 0x7f9c6c072ec0 2024-09-18T07:18:24.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.510+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f9c6c1476a0 con 0x7f9c6c072ec0 2024-09-18T07:18:24.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.709+0000 7f9c728dd640 1 -- 172.21.15.70:0/1369188390 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f9c4c000b90 con 0x7f9c58079990 2024-09-18T07:18:24.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.709+0000 7f9c68ff9640 1 -- 172.21.15.70:0/1369188390 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (unknown 2264484800 0 0) 0x7f9c4c000b90 con 0x7f9c58079990 2024-09-18T07:18:24.712 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 5 to deep-scrub 2024-09-18T07:18:24.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.711+0000 7f9c728dd640 1 -- 172.21.15.70:0/1369188390 >> v1:172.21.15.70:6816/1657990791 conn(0x7f9c58079990 legacy=0x7f9c5807be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.711+0000 7f9c728dd640 1 -- 172.21.15.70:0/1369188390 >> v1:172.21.15.70:6789/0 conn(0x7f9c6c072ec0 legacy=0x7f9c6c133320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.711+0000 7f9c728dd640 1 -- 172.21.15.70:0/1369188390 shutdown_connections 2024-09-18T07:18:24.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.711+0000 7f9c728dd640 1 -- 172.21.15.70:0/1369188390 >> 172.21.15.70:0/1369188390 conn(0x7f9c6c0b64d0 msgr2=0x7f9c6c0702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:24.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.711+0000 7f9c728dd640 1 -- 172.21.15.70:0/1369188390 shutdown_connections 2024-09-18T07:18:24.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.711+0000 7f9c728dd640 1 -- 172.21.15.70:0/1369188390 wait complete. 2024-09-18T07:18:24.725 INFO:tasks.ceph:Scrubbing osd.6 2024-09-18T07:18:24.725 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-18T07:18:24.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.829+0000 7f126aab8640 1 Processor -- start 2024-09-18T07:18:24.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.830+0000 7f126aab8640 1 -- start start 2024-09-18T07:18:24.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.830+0000 7f126aab8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12640b5850 con 0x7f12640b3bd0 2024-09-18T07:18:24.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.830+0000 7f126aab8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1264073400 con 0x7f12640b5470 2024-09-18T07:18:24.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.830+0000 7f126aab8640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1264072260 con 0x7f1264143470 2024-09-18T07:18:24.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.830+0000 7f1263fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f12640b3bd0 0x7f1264143c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57732/0 (socket says 172.21.15.70:57732) 2024-09-18T07:18:24.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.830+0000 7f1263fff640 1 -- 172.21.15.70:0/3634649770 learned_addr learned my addr 172.21.15.70:0/3634649770 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:24.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.830+0000 7f1269055640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f1264143470 0x7f1264144360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48324/0 (socket says 172.21.15.70:48324) 2024-09-18T07:18:24.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.831+0000 7f12637fe640 1 -- 172.21.15.70:0/3634649770 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3654663382 0 0) 0x7f1264072260 con 0x7f1264143470 2024-09-18T07:18:24.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.831+0000 7f12637fe640 1 -- 172.21.15.70:0/3634649770 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1254003520 con 0x7f1264143470 2024-09-18T07:18:24.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.831+0000 7f12637fe640 1 -- 172.21.15.70:0/3634649770 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4083880153 0 0) 0x7f12640b5850 con 0x7f12640b3bd0 2024-09-18T07:18:24.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.831+0000 7f12637fe640 1 -- 172.21.15.70:0/3634649770 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1264072260 con 0x7f12640b3bd0 2024-09-18T07:18:24.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.831+0000 7f12637fe640 1 -- 172.21.15.70:0/3634649770 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1105130452 0 0) 0x7f1264073400 con 0x7f12640b5470 2024-09-18T07:18:24.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.831+0000 7f12637fe640 1 -- 172.21.15.70:0/3634649770 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12640b5850 con 0x7f12640b5470 2024-09-18T07:18:24.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.831+0000 7f12637fe640 1 -- 172.21.15.70:0/3634649770 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1168680524 0 0) 0x7f1254003520 con 0x7f1264143470 2024-09-18T07:18:24.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.831+0000 7f12637fe640 1 -- 172.21.15.70:0/3634649770 >> v1:172.21.15.172:6789/0 conn(0x7f12640b5470 legacy=0x7f1264142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.831+0000 7f12637fe640 1 -- 172.21.15.70:0/3634649770 >> v1:172.21.15.70:6789/0 conn(0x7f12640b3bd0 legacy=0x7f1264143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.831+0000 7f12637fe640 1 -- 172.21.15.70:0/3634649770 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1264144b80 con 0x7f1264143470 2024-09-18T07:18:24.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.831+0000 7f12637fe640 1 -- 172.21.15.70:0/3634649770 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f125c001a10 con 0x7f1264143470 2024-09-18T07:18:24.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.831+0000 7f12637fe640 1 -- 172.21.15.70:0/3634649770 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f125c001c30 con 0x7f1264143470 2024-09-18T07:18:24.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.831+0000 7f12637fe640 1 -- 172.21.15.70:0/3634649770 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f125c0020c0 con 0x7f1264143470 2024-09-18T07:18:24.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.832+0000 7f126aab8640 1 -- 172.21.15.70:0/3634649770 >> v1:172.21.15.70:6790/0 conn(0x7f1264143470 legacy=0x7f1264144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.832+0000 7f126aab8640 1 -- 172.21.15.70:0/3634649770 shutdown_connections 2024-09-18T07:18:24.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.832+0000 7f126aab8640 1 -- 172.21.15.70:0/3634649770 >> 172.21.15.70:0/3634649770 conn(0x7f12640b62f0 msgr2=0x7f12640b66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:24.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.832+0000 7f126aab8640 1 -- 172.21.15.70:0/3634649770 shutdown_connections 2024-09-18T07:18:24.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.832+0000 7f126aab8640 1 -- 172.21.15.70:0/3634649770 wait complete. 2024-09-18T07:18:24.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.833+0000 7f126aab8640 1 Processor -- start 2024-09-18T07:18:24.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.833+0000 7f126aab8640 1 -- start start 2024-09-18T07:18:24.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f126aab8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f126413d840 con 0x7f1264143470 2024-09-18T07:18:24.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f126aab8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f126413db20 con 0x7f12640b5470 2024-09-18T07:18:24.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f126aab8640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f126413bd60 con 0x7f12640b3bd0 2024-09-18T07:18:24.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f1268854640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f12640b3bd0 0x7f1264137910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48336/0 (socket says 172.21.15.70:48336) 2024-09-18T07:18:24.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f1268854640 1 -- 172.21.15.70:0/2564751736 learned_addr learned my addr 172.21.15.70:0/2564751736 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:24.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2086241625 0 0) 0x7f126413d840 con 0x7f1264143470 2024-09-18T07:18:24.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1248003520 con 0x7f1264143470 2024-09-18T07:18:24.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3480395993 0 0) 0x7f126413bd60 con 0x7f12640b3bd0 2024-09-18T07:18:24.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f126413d840 con 0x7f12640b3bd0 2024-09-18T07:18:24.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2895730147 0 0) 0x7f126413db20 con 0x7f12640b5470 2024-09-18T07:18:24.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f126413bd60 con 0x7f12640b5470 2024-09-18T07:18:24.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1619793912 0 0) 0x7f1248003520 con 0x7f1264143470 2024-09-18T07:18:24.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f126413db20 con 0x7f1264143470 2024-09-18T07:18:24.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f125c001f80 con 0x7f1264143470 2024-09-18T07:18:24.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 563810523 0 0) 0x7f126413bd60 con 0x7f12640b5470 2024-09-18T07:18:24.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1248003520 con 0x7f12640b5470 2024-09-18T07:18:24.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f12500010b0 con 0x7f12640b5470 2024-09-18T07:18:24.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2166177286 0 0) 0x7f126413d840 con 0x7f12640b3bd0 2024-09-18T07:18:24.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f126413bd60 con 0x7f12640b3bd0 2024-09-18T07:18:24.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f1258001970 con 0x7f12640b3bd0 2024-09-18T07:18:24.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3634413414 0 0) 0x7f126413db20 con 0x7f1264143470 2024-09-18T07:18:24.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 >> v1:172.21.15.70:6790/0 conn(0x7f12640b3bd0 legacy=0x7f1264137910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 >> v1:172.21.15.172:6789/0 conn(0x7f12640b5470 legacy=0x7f12641d8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12641d9ab0 con 0x7f1264143470 2024-09-18T07:18:24.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f126aab8640 1 -- 172.21.15.70:0/2564751736 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1264138020 con 0x7f1264143470 2024-09-18T07:18:24.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.834+0000 7f126aab8640 1 -- 172.21.15.70:0/2564751736 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f12641478f0 con 0x7f1264143470 2024-09-18T07:18:24.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.835+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f125c001b90 con 0x7f1264143470 2024-09-18T07:18:24.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.835+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f125c004fb0 con 0x7f1264143470 2024-09-18T07:18:24.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.835+0000 7f126aab8640 1 -- 172.21.15.70:0/2564751736 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f1244000f80 con 0x7f1264143470 2024-09-18T07:18:24.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.836+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f125c01dbe0 con 0x7f1264143470 2024-09-18T07:18:24.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.837+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f125c0962a0 con 0x7f1264143470 2024-09-18T07:18:24.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.837+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 --> v1:172.21.15.172:6812/371905845 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1248085480 con 0x7f1248081810 2024-09-18T07:18:24.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.837+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (unknown 2370098492 0 0) 0x7f125c0966b0 con 0x7f1264143470 2024-09-18T07:18:24.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.838+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== osd.6 v1:172.21.15.172:6812/371905845 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f1248085480 con 0x7f1248081810 2024-09-18T07:18:24.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.850+0000 7f126aab8640 1 -- 172.21.15.70:0/2564751736 --> v1:172.21.15.172:6812/371905845 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1244002cd0 con 0x7f1248081810 2024-09-18T07:18:24.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.850+0000 7f12617fa640 1 -- 172.21.15.70:0/2564751736 <== osd.6 v1:172.21.15.172:6812/371905845 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f1244002cd0 con 0x7f1248081810 2024-09-18T07:18:24.853 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-18T07:18:24.854 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T07:18:24.854 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-18T07:18:24.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.851+0000 7f126aab8640 1 -- 172.21.15.70:0/2564751736 >> v1:172.21.15.172:6812/371905845 conn(0x7f1248081810 legacy=0x7f1248083c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.851+0000 7f126aab8640 1 -- 172.21.15.70:0/2564751736 >> v1:172.21.15.70:6816/1657990791 conn(0x7f1248079a20 legacy=0x7f124807bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.851+0000 7f126aab8640 1 -- 172.21.15.70:0/2564751736 >> v1:172.21.15.70:6789/0 conn(0x7f1264143470 legacy=0x7f12641d9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.851+0000 7f1269055640 1 -- 172.21.15.70:0/2564751736 reap_dead start 2024-09-18T07:18:24.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.851+0000 7f126aab8640 1 -- 172.21.15.70:0/2564751736 shutdown_connections 2024-09-18T07:18:24.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.851+0000 7f126aab8640 1 -- 172.21.15.70:0/2564751736 >> 172.21.15.70:0/2564751736 conn(0x7f12640b62f0 msgr2=0x7f1264071050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:24.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.851+0000 7f126aab8640 1 -- 172.21.15.70:0/2564751736 shutdown_connections 2024-09-18T07:18:24.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.851+0000 7f126aab8640 1 -- 172.21.15.70:0/2564751736 wait complete. 2024-09-18T07:18:24.864 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-18T07:18:24.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.968+0000 7f69b0cc6640 1 Processor -- start 2024-09-18T07:18:24.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.968+0000 7f69b0cc6640 1 -- start start 2024-09-18T07:18:24.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.969+0000 7f69b0cc6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69ac073680 con 0x7f69ac13a5f0 2024-09-18T07:18:24.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.969+0000 7f69b0cc6640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69ac071670 con 0x7f69ac0732a0 2024-09-18T07:18:24.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.969+0000 7f69b0cc6640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69ac072990 con 0x7f69ac072ec0 2024-09-18T07:18:24.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.969+0000 7f69aa59c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f69ac072ec0 0x7f69ac139ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48342/0 (socket says 172.21.15.70:48342) 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.969+0000 7f69aa59c640 1 -- 172.21.15.70:0/147281573 learned_addr learned my addr 172.21.15.70:0/147281573 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.970+0000 7f69a959a640 1 -- 172.21.15.70:0/147281573 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 599408511 0 0) 0x7f69a0001040 con 0x7f69ac13a5f0 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.970+0000 7f69a959a640 1 -- 172.21.15.70:0/147281573 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6994003520 con 0x7f69ac13a5f0 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.970+0000 7f69a959a640 1 -- 172.21.15.70:0/147281573 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 647984472 0 0) 0x7f69ac072990 con 0x7f69ac072ec0 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.970+0000 7f69a959a640 1 -- 172.21.15.70:0/147281573 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69a0001040 con 0x7f69ac072ec0 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.970+0000 7f69a959a640 1 -- 172.21.15.70:0/147281573 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 357166428 0 0) 0x7f69ac071670 con 0x7f69ac0732a0 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.970+0000 7f69a959a640 1 -- 172.21.15.70:0/147281573 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69ac072990 con 0x7f69ac0732a0 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.970+0000 7f69a959a640 1 -- 172.21.15.70:0/147281573 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2142067575 0 0) 0x7f69a0001040 con 0x7f69ac072ec0 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.970+0000 7f69a959a640 1 -- 172.21.15.70:0/147281573 >> v1:172.21.15.172:6789/0 conn(0x7f69ac0732a0 legacy=0x7f69ac072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.970+0000 7f69a959a640 1 -- 172.21.15.70:0/147281573 >> v1:172.21.15.70:6789/0 conn(0x7f69ac13a5f0 legacy=0x7f69ac071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.970+0000 7f69a959a640 1 -- 172.21.15.70:0/147281573 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ac144bb0 con 0x7f69ac072ec0 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.970+0000 7f69a959a640 1 -- 172.21.15.70:0/147281573 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f699c001670 con 0x7f69ac072ec0 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.970+0000 7f69a959a640 1 -- 172.21.15.70:0/147281573 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f699c001890 con 0x7f69ac072ec0 2024-09-18T07:18:24.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.970+0000 7f69a959a640 1 -- 172.21.15.70:0/147281573 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f699c001ca0 con 0x7f69ac072ec0 2024-09-18T07:18:24.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.971+0000 7f69b0cc6640 1 -- 172.21.15.70:0/147281573 >> v1:172.21.15.70:6790/0 conn(0x7f69ac072ec0 legacy=0x7f69ac139ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.971+0000 7f69b0cc6640 1 -- 172.21.15.70:0/147281573 shutdown_connections 2024-09-18T07:18:24.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.971+0000 7f69b0cc6640 1 -- 172.21.15.70:0/147281573 >> 172.21.15.70:0/147281573 conn(0x7f69ac0b64d0 msgr2=0x7f69ac0b68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:24.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.971+0000 7f69b0cc6640 1 -- 172.21.15.70:0/147281573 shutdown_connections 2024-09-18T07:18:24.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.971+0000 7f69b0cc6640 1 -- 172.21.15.70:0/147281573 wait complete. 2024-09-18T07:18:24.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.972+0000 7f69b0cc6640 1 Processor -- start 2024-09-18T07:18:24.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.972+0000 7f69b0cc6640 1 -- start start 2024-09-18T07:18:24.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.973+0000 7f69b0cc6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69ac147890 con 0x7f69ac13a5f0 2024-09-18T07:18:24.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.973+0000 7f69b0cc6640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69ac146ad0 con 0x7f69ac072ec0 2024-09-18T07:18:24.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.973+0000 7f69b0cc6640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69ac1dd3b0 con 0x7f69ac0732a0 2024-09-18T07:18:24.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.973+0000 7f69a9d9b640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f69ac0732a0 0x7f69ac13ccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48350/0 (socket says 172.21.15.70:48350) 2024-09-18T07:18:24.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.973+0000 7f69a9d9b640 1 -- 172.21.15.70:0/3526231921 learned_addr learned my addr 172.21.15.70:0/3526231921 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:24.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.973+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3600598502 0 0) 0x7f69ac147890 con 0x7f69ac13a5f0 2024-09-18T07:18:24.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.973+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f698c003520 con 0x7f69ac13a5f0 2024-09-18T07:18:24.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.973+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2168233509 0 0) 0x7f69ac1dd3b0 con 0x7f69ac0732a0 2024-09-18T07:18:24.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.973+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69ac147890 con 0x7f69ac0732a0 2024-09-18T07:18:24.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.973+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3735838638 0 0) 0x7f69ac146ad0 con 0x7f69ac072ec0 2024-09-18T07:18:24.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.973+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69ac1dd3b0 con 0x7f69ac072ec0 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2722612804 0 0) 0x7f698c003520 con 0x7f69ac13a5f0 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f69ac146ad0 con 0x7f69ac13a5f0 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f69a4001c70 con 0x7f69ac13a5f0 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 167361672 0 0) 0x7f69ac1dd3b0 con 0x7f69ac072ec0 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f698c003520 con 0x7f69ac072ec0 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 731356811 0 0) 0x7f69ac147890 con 0x7f69ac0732a0 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f69ac1dd3b0 con 0x7f69ac0732a0 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f699c001a40 con 0x7f69ac072ec0 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f69a00018f0 con 0x7f69ac0732a0 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2872089789 0 0) 0x7f69ac146ad0 con 0x7f69ac13a5f0 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 >> v1:172.21.15.70:6790/0 conn(0x7f69ac0732a0 legacy=0x7f69ac13ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 >> v1:172.21.15.172:6789/0 conn(0x7f69ac072ec0 legacy=0x7f69ac1dc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ac147b00 con 0x7f69ac13a5f0 2024-09-18T07:18:24.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f69b0cc6640 1 -- 172.21.15.70:0/3526231921 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f69ac1459f0 con 0x7f69ac13a5f0 2024-09-18T07:18:24.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f69b0cc6640 1 -- 172.21.15.70:0/3526231921 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f69ac145610 con 0x7f69ac13a5f0 2024-09-18T07:18:24.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f69a4003990 con 0x7f69ac13a5f0 2024-09-18T07:18:24.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.974+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f69a4003ce0 con 0x7f69ac13a5f0 2024-09-18T07:18:24.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.975+0000 7f69b0cc6640 1 -- 172.21.15.70:0/3526231921 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6990005080 con 0x7f69ac13a5f0 2024-09-18T07:18:24.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.976+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f69a401c910 con 0x7f69ac13a5f0 2024-09-18T07:18:24.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.977+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f69a40950b0 con 0x7f69ac13a5f0 2024-09-18T07:18:24.983 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:24.981+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f69a405d910 con 0x7f69ac13a5f0 2024-09-18T07:18:25.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.170+0000 7f69b0cc6640 1 -- 172.21.15.70:0/3526231921 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f6990000b90 con 0x7f698c079af0 2024-09-18T07:18:25.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.170+0000 7f699affd640 1 -- 172.21.15.70:0/3526231921 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (unknown 778718403 0 0) 0x7f6990000b90 con 0x7f698c079af0 2024-09-18T07:18:25.173 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 6 to deep-scrub 2024-09-18T07:18:25.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.172+0000 7f69b0cc6640 1 -- 172.21.15.70:0/3526231921 >> v1:172.21.15.70:6816/1657990791 conn(0x7f698c079af0 legacy=0x7f698c07bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.172+0000 7f69b0cc6640 1 -- 172.21.15.70:0/3526231921 >> v1:172.21.15.70:6789/0 conn(0x7f69ac13a5f0 legacy=0x7f69ac1dcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.172+0000 7f69b0cc6640 1 -- 172.21.15.70:0/3526231921 shutdown_connections 2024-09-18T07:18:25.176 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.172+0000 7f69b0cc6640 1 -- 172.21.15.70:0/3526231921 >> 172.21.15.70:0/3526231921 conn(0x7f69ac0b64d0 msgr2=0x7f69ac137f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:25.176 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.172+0000 7f69b0cc6640 1 -- 172.21.15.70:0/3526231921 shutdown_connections 2024-09-18T07:18:25.176 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.172+0000 7f69b0cc6640 1 -- 172.21.15.70:0/3526231921 wait complete. 2024-09-18T07:18:25.185 INFO:tasks.ceph:Scrubbing osd.7 2024-09-18T07:18:25.186 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-18T07:18:25.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.291+0000 7f2442d52640 1 Processor -- start 2024-09-18T07:18:25.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.291+0000 7f2442d52640 1 -- start start 2024-09-18T07:18:25.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.291+0000 7f2442d52640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f243c0b6410 con 0x7f243c074240 2024-09-18T07:18:25.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.291+0000 7f2442d52640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f243c073330 con 0x7f243c140dd0 2024-09-18T07:18:25.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.291+0000 7f2442d52640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f243c141730 con 0x7f243c073850 2024-09-18T07:18:25.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.291+0000 7f2440aee640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f243c074240 0x7f243c1406c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57768/0 (socket says 172.21.15.70:57768) 2024-09-18T07:18:25.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f2440aee640 1 -- 172.21.15.70:0/2887174613 learned_addr learned my addr 172.21.15.70:0/2887174613 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:25.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f243b7fe640 1 -- 172.21.15.70:0/2887174613 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1190461745 0 0) 0x7f243c0b6410 con 0x7f243c074240 2024-09-18T07:18:25.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f243b7fe640 1 -- 172.21.15.70:0/2887174613 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f242c003520 con 0x7f243c074240 2024-09-18T07:18:25.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f243b7fe640 1 -- 172.21.15.70:0/2887174613 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3455409340 0 0) 0x7f243c141730 con 0x7f243c073850 2024-09-18T07:18:25.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f243b7fe640 1 -- 172.21.15.70:0/2887174613 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f243c0b6410 con 0x7f243c073850 2024-09-18T07:18:25.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f243b7fe640 1 -- 172.21.15.70:0/2887174613 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1018711627 0 0) 0x7f243c073330 con 0x7f243c140dd0 2024-09-18T07:18:25.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f243b7fe640 1 -- 172.21.15.70:0/2887174613 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f243c141730 con 0x7f243c140dd0 2024-09-18T07:18:25.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f243b7fe640 1 -- 172.21.15.70:0/2887174613 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1833015190 0 0) 0x7f242c003520 con 0x7f243c074240 2024-09-18T07:18:25.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f243b7fe640 1 -- 172.21.15.70:0/2887174613 >> v1:172.21.15.70:6790/0 conn(0x7f243c073850 legacy=0x7f243c14bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f243b7fe640 1 -- 172.21.15.70:0/2887174613 >> v1:172.21.15.172:6789/0 conn(0x7f243c140dd0 legacy=0x7f243c139f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f243b7fe640 1 -- 172.21.15.70:0/2887174613 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c0aa440 con 0x7f243c074240 2024-09-18T07:18:25.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f243b7fe640 1 -- 172.21.15.70:0/2887174613 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2430001690 con 0x7f243c074240 2024-09-18T07:18:25.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f243b7fe640 1 -- 172.21.15.70:0/2887174613 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2430001830 con 0x7f243c074240 2024-09-18T07:18:25.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.292+0000 7f243b7fe640 1 -- 172.21.15.70:0/2887174613 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2430001cc0 con 0x7f243c074240 2024-09-18T07:18:25.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.293+0000 7f2442d52640 1 -- 172.21.15.70:0/2887174613 >> v1:172.21.15.70:6789/0 conn(0x7f243c074240 legacy=0x7f243c1406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.293+0000 7f2442d52640 1 -- 172.21.15.70:0/2887174613 shutdown_connections 2024-09-18T07:18:25.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.293+0000 7f2442d52640 1 -- 172.21.15.70:0/2887174613 >> 172.21.15.70:0/2887174613 conn(0x7f243c0b5890 msgr2=0x7f243c0b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:25.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.293+0000 7f2442d52640 1 -- 172.21.15.70:0/2887174613 shutdown_connections 2024-09-18T07:18:25.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.293+0000 7f2442d52640 1 -- 172.21.15.70:0/2887174613 wait complete. 2024-09-18T07:18:25.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.294+0000 7f2442d52640 1 Processor -- start 2024-09-18T07:18:25.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.294+0000 7f2442d52640 1 -- start start 2024-09-18T07:18:25.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.294+0000 7f2442d52640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f243c143ba0 con 0x7f243c074240 2024-09-18T07:18:25.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.294+0000 7f2442d52640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f243c142880 con 0x7f243c073850 2024-09-18T07:18:25.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.294+0000 7f2442d52640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f243c142a50 con 0x7f243c140dd0 2024-09-18T07:18:25.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.294+0000 7f243bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f243c074240 0x7f243c135b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57772/0 (socket says 172.21.15.70:57772) 2024-09-18T07:18:25.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.294+0000 7f24412ef640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f243c140dd0 0x7f243c1d4a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48370/0 (socket says 172.21.15.70:48370) 2024-09-18T07:18:25.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.294+0000 7f243bfff640 1 -- 172.21.15.70:0/4095516026 learned_addr learned my addr 172.21.15.70:0/4095516026 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:25.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 445919693 0 0) 0x7f243c143ba0 con 0x7f243c074240 2024-09-18T07:18:25.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2420003520 con 0x7f243c074240 2024-09-18T07:18:25.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2156749039 0 0) 0x7f243c142a50 con 0x7f243c140dd0 2024-09-18T07:18:25.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f243c143ba0 con 0x7f243c140dd0 2024-09-18T07:18:25.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 182216648 0 0) 0x7f243c142880 con 0x7f243c073850 2024-09-18T07:18:25.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f243c142a50 con 0x7f243c073850 2024-09-18T07:18:25.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 5687372 0 0) 0x7f2420003520 con 0x7f243c074240 2024-09-18T07:18:25.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f243c142880 con 0x7f243c074240 2024-09-18T07:18:25.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2428001970 con 0x7f243c074240 2024-09-18T07:18:25.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2834392261 0 0) 0x7f243c142a50 con 0x7f243c073850 2024-09-18T07:18:25.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2420003520 con 0x7f243c073850 2024-09-18T07:18:25.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3095736548 0 0) 0x7f243c143ba0 con 0x7f243c140dd0 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f243c142a50 con 0x7f243c140dd0 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2430001b80 con 0x7f243c073850 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2434001590 con 0x7f243c140dd0 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2349344828 0 0) 0x7f243c142880 con 0x7f243c074240 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 >> v1:172.21.15.70:6790/0 conn(0x7f243c140dd0 legacy=0x7f243c1d4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 >> v1:172.21.15.172:6789/0 conn(0x7f243c073850 legacy=0x7f243c143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c136250 con 0x7f243c074240 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f2442d52640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f243c13c320 con 0x7f243c074240 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f2442d52640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f243c1d5330 con 0x7f243c074240 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2428003610 con 0x7f243c074240 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.295+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2428003a60 con 0x7f243c074240 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.296+0000 7f2442d52640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f241c000f80 con 0x7f243c074240 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.297+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f2428003c80 con 0x7f243c074240 2024-09-18T07:18:25.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.297+0000 7f243bfff640 1 -- 172.21.15.70:0/4095516026 >> v1:172.21.15.70:6789/0 conn(0x7f243c074240 legacy=0x7f243c135b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-18T07:18:25.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.297+0000 7f243bfff640 1 -- 172.21.15.70:0/4095516026 >> v1:172.21.15.70:6789/0 conn(0x7f243c074240 legacy=0x7f243c135b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-18T07:18:25.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.297+0000 7f243bfff640 1 --1- 172.21.15.70:0/4095516026 >> v1:172.21.15.70:6789/0 conn(0x7f243c074240 0x7f243c135b40 :-1 s=OPENED pgs=267 cs=1 l=1).handle_message read tag failed 2024-09-18T07:18:25.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.297+0000 7f243bfff640 1 --1- 172.21.15.70:0/4095516026 >> v1:172.21.15.70:6789/0 conn(0x7f243c074240 0x7f243c135b40 :-1 s=OPENED pgs=267 cs=1 l=1).fault on lossy channel, failing 2024-09-18T07:18:25.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.297+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 >> v1:172.21.15.70:6789/0 conn(0x7f243c074240 legacy=0x7f243c135b40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-18T07:18:25.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.297+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f243c142880 con 0x7f2420080fd0 2024-09-18T07:18:25.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.297+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f243c143ba0 con 0x7f243c073850 2024-09-18T07:18:25.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.297+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f242008abf0 con 0x7f243c140dd0 2024-09-18T07:18:25.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1558261288 0 0) 0x7f243c142880 con 0x7f2420080fd0 2024-09-18T07:18:25.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6789/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f2420088f30 con 0x7f2420080fd0 2024-09-18T07:18:25.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1929319767 0 0) 0x7f243c143ba0 con 0x7f243c073850 2024-09-18T07:18:25.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.172:6789/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f243c142880 con 0x7f243c073850 2024-09-18T07:18:25.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3141157820 0 0) 0x7f242008abf0 con 0x7f243c140dd0 2024-09-18T07:18:25.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6790/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f243c143ba0 con 0x7f243c140dd0 2024-09-18T07:18:25.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (unknown 2337161074 0 0) 0x7f2420088f30 con 0x7f2420080fd0 2024-09-18T07:18:25.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 >> v1:172.21.15.70:6790/0 conn(0x7f243c140dd0 legacy=0x7f2420087030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 >> v1:172.21.15.172:6789/0 conn(0x7f243c073850 legacy=0x7f242007f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) -- 0x7f242008bdf0 con 0x7f2420080fd0 2024-09-18T07:18:25.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=2) -- 0x7f242008c020 con 0x7f2420080fd0 2024-09-18T07:18:25.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=3) -- 0x7f2420088b90 con 0x7f2420080fd0 2024-09-18T07:18:25.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=4) -- 0x7f2420088d60 con 0x7f2420080fd0 2024-09-18T07:18:25.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2434003970 con 0x7f2420080fd0 2024-09-18T07:18:25.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.298+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2420088d60 con 0x7f2420080fd0 2024-09-18T07:18:25.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.299+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f2434003970 con 0x7f2420080fd0 2024-09-18T07:18:25.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.300+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 6 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f2434095e20 con 0x7f2420080fd0 2024-09-18T07:18:25.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.300+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.172:6800/2423240725 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2420090ae0 con 0x7f243c074240 2024-09-18T07:18:25.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.300+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mon_get_version_reply(handle=2 version=26) ==== 24+0+0 (unknown 1182327903 0 0) 0x7f2434001590 con 0x7f2420080fd0 2024-09-18T07:18:25.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.300+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 8 ==== mon_get_version_reply(handle=3 version=26) ==== 24+0+0 (unknown 4231472849 0 0) 0x7f24340960c0 con 0x7f2420080fd0 2024-09-18T07:18:25.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.300+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=4 version=26) ==== 24+0+0 (unknown 3589075048 0 0) 0x7f24340963c0 con 0x7f2420080fd0 2024-09-18T07:18:25.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.301+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== osd.7 v1:172.21.15.172:6800/2423240725 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f2420090ae0 con 0x7f243c074240 2024-09-18T07:18:25.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.313+0000 7f2442d52640 1 -- 172.21.15.70:0/4095516026 --> v1:172.21.15.172:6800/2423240725 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f241c002cd0 con 0x7f243c074240 2024-09-18T07:18:25.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.313+0000 7f24397fa640 1 -- 172.21.15.70:0/4095516026 <== osd.7 v1:172.21.15.172:6800/2423240725 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f241c002cd0 con 0x7f243c074240 2024-09-18T07:18:25.316 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-18T07:18:25.316 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T07:18:25.316 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-18T07:18:25.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.314+0000 7f2442d52640 1 -- 172.21.15.70:0/4095516026 >> v1:172.21.15.172:6800/2423240725 conn(0x7f243c074240 legacy=0x7f242008f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.314+0000 7f2442d52640 1 -- 172.21.15.70:0/4095516026 >> v1:172.21.15.70:6816/1657990791 conn(0x7f2420079a30 legacy=0x7f242007bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.314+0000 7f2442d52640 1 -- 172.21.15.70:0/4095516026 >> v1:172.21.15.70:6789/0 conn(0x7f2420080fd0 legacy=0x7f2420083460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.314+0000 7f24412ef640 1 -- 172.21.15.70:0/4095516026 reap_dead start 2024-09-18T07:18:25.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.314+0000 7f2442d52640 1 -- 172.21.15.70:0/4095516026 shutdown_connections 2024-09-18T07:18:25.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.314+0000 7f2442d52640 1 -- 172.21.15.70:0/4095516026 >> 172.21.15.70:0/4095516026 conn(0x7f243c0b5890 msgr2=0x7f243c136e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:25.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.314+0000 7f2442d52640 1 -- 172.21.15.70:0/4095516026 shutdown_connections 2024-09-18T07:18:25.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.314+0000 7f2442d52640 1 -- 172.21.15.70:0/4095516026 wait complete. 2024-09-18T07:18:25.327 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-18T07:18:25.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.435+0000 7f29a15cd640 1 Processor -- start 2024-09-18T07:18:25.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.436+0000 7f29a15cd640 1 -- start start 2024-09-18T07:18:25.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.436+0000 7f29a15cd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f299c0b64f0 con 0x7f299c073910 2024-09-18T07:18:25.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.436+0000 7f29a15cd640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f299c0ab5c0 con 0x7f299c0b6110 2024-09-18T07:18:25.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.436+0000 7f29a15cd640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f299c0733f0 con 0x7f299c13a7a0 2024-09-18T07:18:25.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.436+0000 7f299a7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f299c073910 0x7f299c143bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57776/0 (socket says 172.21.15.70:57776) 2024-09-18T07:18:25.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.436+0000 7f299a7fc640 1 -- 172.21.15.70:0/730196260 learned_addr learned my addr 172.21.15.70:0/730196260 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:25.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.436+0000 7f299b7fe640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f299c13a7a0 0x7f299c14bf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48378/0 (socket says 172.21.15.70:48378) 2024-09-18T07:18:25.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.436+0000 7f2999ffb640 1 -- 172.21.15.70:0/730196260 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 791845934 0 0) 0x7f299c0b64f0 con 0x7f299c073910 2024-09-18T07:18:25.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f2999ffb640 1 -- 172.21.15.70:0/730196260 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2984003520 con 0x7f299c073910 2024-09-18T07:18:25.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f2999ffb640 1 -- 172.21.15.70:0/730196260 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1559593164 0 0) 0x7f299c0733f0 con 0x7f299c13a7a0 2024-09-18T07:18:25.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f2999ffb640 1 -- 172.21.15.70:0/730196260 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f299c0b64f0 con 0x7f299c13a7a0 2024-09-18T07:18:25.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f2999ffb640 1 -- 172.21.15.70:0/730196260 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2061614713 0 0) 0x7f299c0ab5c0 con 0x7f299c0b6110 2024-09-18T07:18:25.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f2999ffb640 1 -- 172.21.15.70:0/730196260 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f299c0733f0 con 0x7f299c0b6110 2024-09-18T07:18:25.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f2999ffb640 1 -- 172.21.15.70:0/730196260 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1944098868 0 0) 0x7f2984003520 con 0x7f299c073910 2024-09-18T07:18:25.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f2999ffb640 1 -- 172.21.15.70:0/730196260 >> v1:172.21.15.70:6790/0 conn(0x7f299c13a7a0 legacy=0x7f299c14bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f2999ffb640 1 -- 172.21.15.70:0/730196260 >> v1:172.21.15.172:6789/0 conn(0x7f299c0b6110 legacy=0x7f299c13a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f2999ffb640 1 -- 172.21.15.70:0/730196260 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c0743d0 con 0x7f299c073910 2024-09-18T07:18:25.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f2999ffb640 1 -- 172.21.15.70:0/730196260 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2990001690 con 0x7f299c073910 2024-09-18T07:18:25.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f2999ffb640 1 -- 172.21.15.70:0/730196260 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f29900018b0 con 0x7f299c073910 2024-09-18T07:18:25.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f2999ffb640 1 -- 172.21.15.70:0/730196260 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2990001d00 con 0x7f299c073910 2024-09-18T07:18:25.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f29a15cd640 1 -- 172.21.15.70:0/730196260 >> v1:172.21.15.70:6789/0 conn(0x7f299c073910 legacy=0x7f299c143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f29a15cd640 1 -- 172.21.15.70:0/730196260 shutdown_connections 2024-09-18T07:18:25.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f29a15cd640 1 -- 172.21.15.70:0/730196260 >> 172.21.15.70:0/730196260 conn(0x7f299c076bb0 msgr2=0x7f299c0b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:25.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f29a15cd640 1 -- 172.21.15.70:0/730196260 shutdown_connections 2024-09-18T07:18:25.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.437+0000 7f29a15cd640 1 -- 172.21.15.70:0/730196260 wait complete. 2024-09-18T07:18:25.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.438+0000 7f29a15cd640 1 Processor -- start 2024-09-18T07:18:25.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.438+0000 7f29a15cd640 1 -- start start 2024-09-18T07:18:25.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.439+0000 7f29a15cd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f299c136410 con 0x7f299c13a7a0 2024-09-18T07:18:25.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.439+0000 7f29a15cd640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f299c1d9830 con 0x7f299c0b6110 2024-09-18T07:18:25.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.439+0000 7f29a15cd640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f299c0b57d0 con 0x7f299c0b4e20 2024-09-18T07:18:25.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.439+0000 7f299a7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f299c13a7a0 0x7f299c1355f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57790/0 (socket says 172.21.15.70:57790) 2024-09-18T07:18:25.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.439+0000 7f299a7fc640 1 -- 172.21.15.70:0/3799491979 learned_addr learned my addr 172.21.15.70:0/3799491979 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:25.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.439+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3686193787 0 0) 0x7f299c136410 con 0x7f299c13a7a0 2024-09-18T07:18:25.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.439+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2988003520 con 0x7f299c13a7a0 2024-09-18T07:18:25.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.439+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 518730711 0 0) 0x7f299c0b57d0 con 0x7f299c0b4e20 2024-09-18T07:18:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.439+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f299c136410 con 0x7f299c0b4e20 2024-09-18T07:18:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.439+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2769113442 0 0) 0x7f299c1d9830 con 0x7f299c0b6110 2024-09-18T07:18:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.439+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f299c0b57d0 con 0x7f299c0b6110 2024-09-18T07:18:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1929980437 0 0) 0x7f2988003520 con 0x7f299c13a7a0 2024-09-18T07:18:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f299c1d9830 con 0x7f299c13a7a0 2024-09-18T07:18:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2990001c00 con 0x7f299c13a7a0 2024-09-18T07:18:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1439730679 0 0) 0x7f299c136410 con 0x7f299c0b4e20 2024-09-18T07:18:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2988003520 con 0x7f299c0b4e20 2024-09-18T07:18:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2994001aa0 con 0x7f299c0b4e20 2024-09-18T07:18:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1281861433 0 0) 0x7f299c0b57d0 con 0x7f299c0b6110 2024-09-18T07:18:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f299c136410 con 0x7f299c0b6110 2024-09-18T07:18:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f298c001830 con 0x7f299c0b6110 2024-09-18T07:18:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3984106187 0 0) 0x7f299c1d9830 con 0x7f299c13a7a0 2024-09-18T07:18:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 >> v1:172.21.15.70:6790/0 conn(0x7f299c0b4e20 legacy=0x7f299c135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 >> v1:172.21.15.172:6789/0 conn(0x7f299c0b6110 legacy=0x7f299c1dbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c0b59a0 con 0x7f299c13a7a0 2024-09-18T07:18:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29a15cd640 1 -- 172.21.15.70:0/3799491979 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f299c1dc420 con 0x7f299c13a7a0 2024-09-18T07:18:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29a15cd640 1 -- 172.21.15.70:0/3799491979 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f299c0b47d0 con 0x7f299c13a7a0 2024-09-18T07:18:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2990001810 con 0x7f299c13a7a0 2024-09-18T07:18:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.440+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f2990004e10 con 0x7f299c13a7a0 2024-09-18T07:18:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.441+0000 7f29a15cd640 1 -- 172.21.15.70:0/3799491979 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f297c005080 con 0x7f299c13a7a0 2024-09-18T07:18:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.442+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f299001da40 con 0x7f299c13a7a0 2024-09-18T07:18:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.442+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f2990095ff0 con 0x7f299c13a7a0 2024-09-18T07:18:25.448 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.446+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f299005e930 con 0x7f299c13a7a0 2024-09-18T07:18:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.682+0000 7f29a15cd640 1 -- 172.21.15.70:0/3799491979 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f297c000b90 con 0x7f29880799b0 2024-09-18T07:18:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.683+0000 7f29837fe640 1 -- 172.21.15.70:0/3799491979 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (unknown 1239724482 0 0) 0x7f297c000b90 con 0x7f29880799b0 2024-09-18T07:18:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 7 to deep-scrub 2024-09-18T07:18:25.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.685+0000 7f29a15cd640 1 -- 172.21.15.70:0/3799491979 >> v1:172.21.15.70:6816/1657990791 conn(0x7f29880799b0 legacy=0x7f298807be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.685+0000 7f29a15cd640 1 -- 172.21.15.70:0/3799491979 >> v1:172.21.15.70:6789/0 conn(0x7f299c13a7a0 legacy=0x7f299c1355f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.685+0000 7f29a15cd640 1 -- 172.21.15.70:0/3799491979 shutdown_connections 2024-09-18T07:18:25.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.685+0000 7f29a15cd640 1 -- 172.21.15.70:0/3799491979 >> 172.21.15.70:0/3799491979 conn(0x7f299c076bb0 msgr2=0x7f299c1405b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:25.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.685+0000 7f29a15cd640 1 -- 172.21.15.70:0/3799491979 shutdown_connections 2024-09-18T07:18:25.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.685+0000 7f29a15cd640 1 -- 172.21.15.70:0/3799491979 wait complete. 2024-09-18T07:18:25.699 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T07:18:25.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.793+0000 7f83eb500640 1 Processor -- start 2024-09-18T07:18:25.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.793+0000 7f83eb500640 1 -- start start 2024-09-18T07:18:25.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83eb500640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83e41441f0 con 0x7f83e4072ea0 2024-09-18T07:18:25.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83eb500640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83e40aae80 con 0x7f83e413f9f0 2024-09-18T07:18:25.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83eb500640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83e4140f40 con 0x7f83e4072ac0 2024-09-18T07:18:25.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83e929c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f83e4072ac0 0x7f83e4143ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48388/0 (socket says 172.21.15.70:48388) 2024-09-18T07:18:25.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83e929c640 1 -- 172.21.15.70:0/4184649508 learned_addr learned my addr 172.21.15.70:0/4184649508 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:25.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83dbfff640 1 -- 172.21.15.70:0/4184649508 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3251741936 0 0) 0x7f83e4140f40 con 0x7f83e4072ac0 2024-09-18T07:18:25.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83dbfff640 1 -- 172.21.15.70:0/4184649508 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83d0004630 con 0x7f83e4072ac0 2024-09-18T07:18:25.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83dbfff640 1 -- 172.21.15.70:0/4184649508 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1054332909 0 0) 0x7f83e41441f0 con 0x7f83e4072ea0 2024-09-18T07:18:25.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83dbfff640 1 -- 172.21.15.70:0/4184649508 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83e4140f40 con 0x7f83e4072ea0 2024-09-18T07:18:25.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83dbfff640 1 -- 172.21.15.70:0/4184649508 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3496399086 0 0) 0x7f83e40aae80 con 0x7f83e413f9f0 2024-09-18T07:18:25.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83dbfff640 1 -- 172.21.15.70:0/4184649508 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83e41441f0 con 0x7f83e413f9f0 2024-09-18T07:18:25.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83dbfff640 1 -- 172.21.15.70:0/4184649508 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3021976318 0 0) 0x7f83d0004630 con 0x7f83e4072ac0 2024-09-18T07:18:25.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83dbfff640 1 -- 172.21.15.70:0/4184649508 >> v1:172.21.15.172:6789/0 conn(0x7f83e413f9f0 legacy=0x7f83e414b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83dbfff640 1 -- 172.21.15.70:0/4184649508 >> v1:172.21.15.70:6789/0 conn(0x7f83e4072ea0 legacy=0x7f83e413f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83dbfff640 1 -- 172.21.15.70:0/4184649508 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e4073f80 con 0x7f83e4072ac0 2024-09-18T07:18:25.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.794+0000 7f83dbfff640 1 -- 172.21.15.70:0/4184649508 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f83e0001690 con 0x7f83e4072ac0 2024-09-18T07:18:25.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.795+0000 7f83dbfff640 1 -- 172.21.15.70:0/4184649508 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f83e0001830 con 0x7f83e4072ac0 2024-09-18T07:18:25.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.795+0000 7f83dbfff640 1 -- 172.21.15.70:0/4184649508 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f83e0001cc0 con 0x7f83e4072ac0 2024-09-18T07:18:25.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.795+0000 7f83eb500640 1 -- 172.21.15.70:0/4184649508 >> v1:172.21.15.70:6790/0 conn(0x7f83e4072ac0 legacy=0x7f83e4143ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.795+0000 7f83eb500640 1 -- 172.21.15.70:0/4184649508 shutdown_connections 2024-09-18T07:18:25.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.795+0000 7f83eb500640 1 -- 172.21.15.70:0/4184649508 >> 172.21.15.70:0/4184649508 conn(0x7f83e40b5790 msgr2=0x7f83e40b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:25.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.795+0000 7f83eb500640 1 -- 172.21.15.70:0/4184649508 shutdown_connections 2024-09-18T07:18:25.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.795+0000 7f83eb500640 1 -- 172.21.15.70:0/4184649508 wait complete. 2024-09-18T07:18:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.796+0000 7f83eb500640 1 Processor -- start 2024-09-18T07:18:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.796+0000 7f83eb500640 1 -- start start 2024-09-18T07:18:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83eb500640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83e40ad7f0 con 0x7f83e413f9f0 2024-09-18T07:18:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83eb500640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83e40adad0 con 0x7f83e4072ea0 2024-09-18T07:18:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83eb500640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83e41d43f0 con 0x7f83e4072ac0 2024-09-18T07:18:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83e929c640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f83e4072ac0 0x7f83e41d4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48396/0 (socket says 172.21.15.70:48396) 2024-09-18T07:18:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83e929c640 1 -- 172.21.15.70:0/974890430 learned_addr learned my addr 172.21.15.70:0/974890430 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2201239748 0 0) 0x7f83e40ad7f0 con 0x7f83e413f9f0 2024-09-18T07:18:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83c8003520 con 0x7f83e413f9f0 2024-09-18T07:18:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3909835724 0 0) 0x7f83e41d43f0 con 0x7f83e4072ac0 2024-09-18T07:18:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83e40ad7f0 con 0x7f83e4072ac0 2024-09-18T07:18:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3882420634 0 0) 0x7f83c8003520 con 0x7f83e413f9f0 2024-09-18T07:18:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f83e41d43f0 con 0x7f83e413f9f0 2024-09-18T07:18:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f83dc001bd0 con 0x7f83e413f9f0 2024-09-18T07:18:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2295247264 0 0) 0x7f83e40ad7f0 con 0x7f83e4072ac0 2024-09-18T07:18:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f83c8003520 con 0x7f83e4072ac0 2024-09-18T07:18:25.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f83e0001b80 con 0x7f83e4072ac0 2024-09-18T07:18:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4216889572 0 0) 0x7f83e40adad0 con 0x7f83e4072ea0 2024-09-18T07:18:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83e40ad7f0 con 0x7f83e4072ea0 2024-09-18T07:18:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 986263878 0 0) 0x7f83e41d43f0 con 0x7f83e413f9f0 2024-09-18T07:18:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 >> v1:172.21.15.70:6790/0 conn(0x7f83e4072ac0 legacy=0x7f83e41d4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 >> v1:172.21.15.172:6789/0 conn(0x7f83e4072ea0 legacy=0x7f83e413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e41d5020 con 0x7f83e413f9f0 2024-09-18T07:18:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.797+0000 7f83eb500640 1 -- 172.21.15.70:0/974890430 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f83e413b0a0 con 0x7f83e413f9f0 2024-09-18T07:18:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.798+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f83dc003990 con 0x7f83e413f9f0 2024-09-18T07:18:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.798+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f83dc003ce0 con 0x7f83e413f9f0 2024-09-18T07:18:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.798+0000 7f83eb500640 1 -- 172.21.15.70:0/974890430 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f83e40ab790 con 0x7f83e413f9f0 2024-09-18T07:18:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.799+0000 7f83eb500640 1 -- 172.21.15.70:0/974890430 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e41d45c0 con 0x7f83e413f9f0 2024-09-18T07:18:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.799+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f83dc01cab0 con 0x7f83e413f9f0 2024-09-18T07:18:25.810 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.800+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f83dc095300 con 0x7f83e413f9f0 2024-09-18T07:18:25.810 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.804+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f83dc05db40 con 0x7f83e413f9f0 2024-09-18T07:18:25.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.986+0000 7f83eb500640 1 -- 172.21.15.70:0/974890430 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83e4069b00 con 0x7f83c80798d0 2024-09-18T07:18:25.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.988+0000 7f83d9ffb640 1 -- 172.21.15.70:0/974890430 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117429 (unknown 2965378022 0 1332197294) 0x7f83e4069b00 con 0x7f83c80798d0 2024-09-18T07:18:25.992 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:18:25.993 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-18T07:18:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.990+0000 7f83eb500640 1 -- 172.21.15.70:0/974890430 >> v1:172.21.15.70:6816/1657990791 conn(0x7f83c80798d0 legacy=0x7f83c807bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.990+0000 7f83eb500640 1 -- 172.21.15.70:0/974890430 >> v1:172.21.15.70:6789/0 conn(0x7f83e413f9f0 legacy=0x7f83e41dfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.990+0000 7f83eb500640 1 -- 172.21.15.70:0/974890430 shutdown_connections 2024-09-18T07:18:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.990+0000 7f83eb500640 1 -- 172.21.15.70:0/974890430 >> 172.21.15.70:0/974890430 conn(0x7f83e40b5790 msgr2=0x7f83e40711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.990+0000 7f83eb500640 1 -- 172.21.15.70:0/974890430 shutdown_connections 2024-09-18T07:18:25.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:25.990+0000 7f83eb500640 1 -- 172.21.15.70:0/974890430 wait complete. 2024-09-18T07:18:26.004 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-18T07:18:25.497379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3582288683,"num_objects":885,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":2641,"num_read_kb":2646,"num_write":165445,"num_write_kb":5660671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":168363,"ondisk_log_size":168363,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8574172,"kb_used_data":6707784,"kb_used_omap":12,"kb_used_meta":1866227,"kb_avail":746400548,"statfs":{"total":773094113280,"available":764314161152,"internally_reserved":0,"allocated":6868770816,"data_stored":6863217094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1911017008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8720,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":66,"num_read_kb":71,"num_write":17,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001898"},"pg_stats":[{"pgid":"3.1f","version":"24'3617","reported_seq":3637,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.274365+0000","last_change":"2024-09-18T07:14:22.286661+0000","last_active":"2024-09-18T07:18:19.274365+0000","last_peered":"2024-09-18T07:18:19.274365+0000","last_clean":"2024-09-18T07:18:19.274365+0000","last_became_active":"2024-09-18T07:14:22.286401+0000","last_became_peered":"2024-09-18T07:14:22.286401+0000","last_unstale":"2024-09-18T07:18:19.274365+0000","last_undegraded":"2024-09-18T07:18:19.274365+0000","last_fullsized":"2024-09-18T07:18:19.274365+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3617,"log_dups_size":0,"ondisk_log_size":3617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:26:38.664961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95160672,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":3623,"num_write_kb":151525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"26'6415","reported_seq":6535,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.284390+0000","last_change":"2024-09-18T07:14:22.284740+0000","last_active":"2024-09-18T07:18:19.284390+0000","last_peered":"2024-09-18T07:18:19.284390+0000","last_clean":"2024-09-18T07:18:19.284390+0000","last_became_active":"2024-09-18T07:14:22.284487+0000","last_became_peered":"2024-09-18T07:14:22.284487+0000","last_unstale":"2024-09-18T07:18:19.284390+0000","last_undegraded":"2024-09-18T07:18:19.284390+0000","last_fullsized":"2024-09-18T07:18:19.284390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":6415,"log_dups_size":0,"ondisk_log_size":6415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377808,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":6397,"num_write_kb":190429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"26'7588","reported_seq":7639,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.274563+0000","last_change":"2024-09-18T07:14:22.288902+0000","last_active":"2024-09-18T07:18:19.274563+0000","last_peered":"2024-09-18T07:18:19.274563+0000","last_clean":"2024-09-18T07:18:19.274563+0000","last_became_active":"2024-09-18T07:14:22.288441+0000","last_became_peered":"2024-09-18T07:14:22.288441+0000","last_unstale":"2024-09-18T07:18:19.274563+0000","last_undegraded":"2024-09-18T07:18:19.274563+0000","last_fullsized":"2024-09-18T07:18:19.274563+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7588,"log_dups_size":0,"ondisk_log_size":7588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:04:30.016419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167903232,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":127,"num_read_kb":103,"num_write":7620,"num_write_kb":243849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"24'3255","reported_seq":3292,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.274351+0000","last_change":"2024-09-18T07:14:22.288842+0000","last_active":"2024-09-18T07:18:19.274351+0000","last_peered":"2024-09-18T07:18:19.274351+0000","last_clean":"2024-09-18T07:18:19.274351+0000","last_became_active":"2024-09-18T07:14:22.287853+0000","last_became_peered":"2024-09-18T07:14:22.287853+0000","last_unstale":"2024-09-18T07:18:19.274351+0000","last_undegraded":"2024-09-18T07:18:19.274351+0000","last_fullsized":"2024-09-18T07:18:19.274351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3255,"log_dups_size":0,"ondisk_log_size":3255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:14.436120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88211456,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":79,"num_read_kb":61,"num_write":3296,"num_write_kb":156739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"26'3831","reported_seq":3852,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.285277+0000","last_change":"2024-09-18T07:14:22.286284+0000","last_active":"2024-09-18T07:18:19.285277+0000","last_peered":"2024-09-18T07:18:19.285277+0000","last_clean":"2024-09-18T07:18:19.285277+0000","last_became_active":"2024-09-18T07:14:22.286018+0000","last_became_peered":"2024-09-18T07:14:22.286018+0000","last_unstale":"2024-09-18T07:18:19.285277+0000","last_undegraded":"2024-09-18T07:18:19.285277+0000","last_fullsized":"2024-09-18T07:18:19.285277+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3831,"log_dups_size":0,"ondisk_log_size":3831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:03:53.111506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91650400,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":3788,"num_write_kb":184441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"26'5566","reported_seq":5588,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.236527+0000","last_change":"2024-09-18T07:14:22.280275+0000","last_active":"2024-09-18T07:18:19.236527+0000","last_peered":"2024-09-18T07:18:19.236527+0000","last_clean":"2024-09-18T07:18:19.236527+0000","last_became_active":"2024-09-18T07:14:22.279826+0000","last_became_peered":"2024-09-18T07:14:22.279826+0000","last_unstale":"2024-09-18T07:18:19.236527+0000","last_undegraded":"2024-09-18T07:18:19.236527+0000","last_fullsized":"2024-09-18T07:18:19.236527+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5566,"log_dups_size":0,"ondisk_log_size":5566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:19:40.864770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":15,"num_read_kb":75,"num_write":5550,"num_write_kb":155179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"26'4745","reported_seq":4779,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.279786+0000","last_change":"2024-09-18T07:14:22.280164+0000","last_active":"2024-09-18T07:18:19.279786+0000","last_peered":"2024-09-18T07:18:19.279786+0000","last_clean":"2024-09-18T07:18:19.279786+0000","last_became_active":"2024-09-18T07:14:22.279885+0000","last_became_peered":"2024-09-18T07:14:22.279885+0000","last_unstale":"2024-09-18T07:18:19.279786+0000","last_undegraded":"2024-09-18T07:18:19.279786+0000","last_fullsized":"2024-09-18T07:18:19.279786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4745,"log_dups_size":0,"ondisk_log_size":4745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:08:22.395662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":4713,"num_write_kb":147181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"25'8007","reported_seq":8026,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.271475+0000","last_change":"2024-09-18T07:14:22.283501+0000","last_active":"2024-09-18T07:18:19.271475+0000","last_peered":"2024-09-18T07:18:19.271475+0000","last_clean":"2024-09-18T07:18:19.271475+0000","last_became_active":"2024-09-18T07:14:22.283231+0000","last_became_peered":"2024-09-18T07:14:22.283231+0000","last_unstale":"2024-09-18T07:18:19.271475+0000","last_undegraded":"2024-09-18T07:18:19.271475+0000","last_fullsized":"2024-09-18T07:18:19.271475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":8007,"log_dups_size":0,"ondisk_log_size":8007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:49:47.239612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125895176,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":8013,"num_write_kb":214525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"25'7235","reported_seq":7257,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.274630+0000","last_change":"2024-09-18T07:14:22.286153+0000","last_active":"2024-09-18T07:18:19.274630+0000","last_peered":"2024-09-18T07:18:19.274630+0000","last_clean":"2024-09-18T07:18:19.274630+0000","last_became_active":"2024-09-18T07:14:22.285884+0000","last_became_peered":"2024-09-18T07:14:22.285884+0000","last_unstale":"2024-09-18T07:18:19.274630+0000","last_undegraded":"2024-09-18T07:18:19.274630+0000","last_fullsized":"2024-09-18T07:18:19.274630+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7235,"log_dups_size":0,"ondisk_log_size":7235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:36:41.719570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135659520,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":7238,"num_write_kb":200827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"26'3409","reported_seq":3435,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.274525+0000","last_change":"2024-09-18T07:14:22.289679+0000","last_active":"2024-09-18T07:18:19.274525+0000","last_peered":"2024-09-18T07:18:19.274525+0000","last_clean":"2024-09-18T07:18:19.274525+0000","last_became_active":"2024-09-18T07:14:22.289372+0000","last_became_peered":"2024-09-18T07:14:22.289372+0000","last_unstale":"2024-09-18T07:18:19.274525+0000","last_undegraded":"2024-09-18T07:18:19.274525+0000","last_fullsized":"2024-09-18T07:18:19.274525+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3409,"log_dups_size":0,"ondisk_log_size":3409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:59:31.222383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112003944,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":22,"num_read_kb":143,"num_write":3412,"num_write_kb":151531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"25'8025","reported_seq":8046,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.246375+0000","last_change":"2024-09-18T07:14:22.288942+0000","last_active":"2024-09-18T07:18:19.246375+0000","last_peered":"2024-09-18T07:18:19.246375+0000","last_clean":"2024-09-18T07:18:19.246375+0000","last_became_active":"2024-09-18T07:14:22.288634+0000","last_became_peered":"2024-09-18T07:14:22.288634+0000","last_unstale":"2024-09-18T07:18:19.246375+0000","last_undegraded":"2024-09-18T07:18:19.246375+0000","last_fullsized":"2024-09-18T07:18:19.246375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":8025,"log_dups_size":0,"ondisk_log_size":8025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:34:15.636211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8027,"num_write_kb":201731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"26'4874","reported_seq":4893,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.280042+0000","last_change":"2024-09-18T07:14:22.291854+0000","last_active":"2024-09-18T07:18:19.280042+0000","last_peered":"2024-09-18T07:18:19.280042+0000","last_clean":"2024-09-18T07:18:19.280042+0000","last_became_active":"2024-09-18T07:14:22.291551+0000","last_became_peered":"2024-09-18T07:14:22.291551+0000","last_unstale":"2024-09-18T07:18:19.280042+0000","last_undegraded":"2024-09-18T07:18:19.280042+0000","last_fullsized":"2024-09-18T07:18:19.280042+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4874,"log_dups_size":0,"ondisk_log_size":4874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:54:13.729656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303680,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4826,"num_write_kb":160133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.276036+0000","last_change":"2024-09-18T07:14:15.808642+0000","last_active":"2024-09-18T07:18:19.276036+0000","last_peered":"2024-09-18T07:18:19.276036+0000","last_clean":"2024-09-18T07:18:19.276036+0000","last_became_active":"2024-09-18T07:14:13.251609+0000","last_became_peered":"2024-09-18T07:14:13.251609+0000","last_unstale":"2024-09-18T07:18:19.276036+0000","last_undegraded":"2024-09-18T07:18:19.276036+0000","last_fullsized":"2024-09-18T07:18:19.276036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:01.748551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040903899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'7912","reported_seq":7940,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:24.631667+0000","last_change":"2024-09-18T07:18:24.631667+0000","last_active":"2024-09-18T07:18:24.631667+0000","last_peered":"2024-09-18T07:18:24.631667+0000","last_clean":"2024-09-18T07:18:24.631667+0000","last_became_active":"2024-09-18T07:14:22.282926+0000","last_became_peered":"2024-09-18T07:14:22.282926+0000","last_unstale":"2024-09-18T07:18:24.631667+0000","last_undegraded":"2024-09-18T07:18:24.631667+0000","last_fullsized":"2024-09-18T07:18:24.631667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'7912","last_scrub_stamp":"2024-09-18T07:18:24.631601+0000","last_deep_scrub":"26'7912","last_deep_scrub_stamp":"2024-09-18T07:18:24.631601+0000","last_clean_scrub_stamp":"2024-09-18T07:18:24.631601+0000","objects_scrubbed":34,"log_size":7912,"log_dups_size":0,"ondisk_log_size":7912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:29:00.658275+0000","scrub_duration":252,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141926400,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7854,"num_write_kb":222187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:24.411880+0000","last_change":"2024-09-18T07:18:24.411880+0000","last_active":"2024-09-18T07:18:24.411880+0000","last_peered":"2024-09-18T07:18:24.411880+0000","last_clean":"2024-09-18T07:18:24.411880+0000","last_became_active":"2024-09-18T07:14:13.251585+0000","last_became_peered":"2024-09-18T07:14:13.251585+0000","last_unstale":"2024-09-18T07:18:24.411880+0000","last_undegraded":"2024-09-18T07:18:24.411880+0000","last_fullsized":"2024-09-18T07:18:24.411880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:18:24.411805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:18:24.411805+0000","last_clean_scrub_stamp":"2024-09-18T07:18:24.411805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:55.857411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039885799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"26'4222","reported_seq":4250,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.283237+0000","last_change":"2024-09-18T07:14:22.288077+0000","last_active":"2024-09-18T07:18:19.283237+0000","last_peered":"2024-09-18T07:18:19.283237+0000","last_clean":"2024-09-18T07:18:19.283237+0000","last_became_active":"2024-09-18T07:14:22.287843+0000","last_became_peered":"2024-09-18T07:14:22.287843+0000","last_unstale":"2024-09-18T07:18:19.283237+0000","last_undegraded":"2024-09-18T07:18:19.283237+0000","last_fullsized":"2024-09-18T07:18:19.283237+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4222,"log_dups_size":0,"ondisk_log_size":4222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:13.075300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141164544,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":4202,"num_write_kb":175065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.246833+0000","last_change":"2024-09-18T07:14:15.603034+0000","last_active":"2024-09-18T07:18:19.246833+0000","last_peered":"2024-09-18T07:18:19.246833+0000","last_clean":"2024-09-18T07:18:19.246833+0000","last_became_active":"2024-09-18T07:14:13.957168+0000","last_became_peered":"2024-09-18T07:14:13.957168+0000","last_unstale":"2024-09-18T07:18:19.246833+0000","last_undegraded":"2024-09-18T07:18:19.246833+0000","last_fullsized":"2024-09-18T07:18:19.246833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:57:48.846502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048630800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"25'7512","reported_seq":7530,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.236679+0000","last_change":"2024-09-18T07:14:22.475409+0000","last_active":"2024-09-18T07:18:19.236679+0000","last_peered":"2024-09-18T07:18:19.236679+0000","last_clean":"2024-09-18T07:18:19.236679+0000","last_became_active":"2024-09-18T07:14:22.475086+0000","last_became_peered":"2024-09-18T07:14:22.475086+0000","last_unstale":"2024-09-18T07:18:19.236679+0000","last_undegraded":"2024-09-18T07:18:19.236679+0000","last_fullsized":"2024-09-18T07:18:19.236679+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7512,"log_dups_size":0,"ondisk_log_size":7512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:33:15.601379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7506,"num_write_kb":189960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:23.367033+0000","last_change":"2024-09-18T07:18:23.366985+0000","last_active":"2024-09-18T07:18:23.367033+0000","last_peered":"2024-09-18T07:18:23.367033+0000","last_clean":"2024-09-18T07:18:23.367033+0000","last_became_active":"2024-09-18T07:14:13.956929+0000","last_became_peered":"2024-09-18T07:14:13.956929+0000","last_unstale":"2024-09-18T07:18:23.367033+0000","last_undegraded":"2024-09-18T07:18:23.367033+0000","last_fullsized":"2024-09-18T07:18:23.367033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:18:23.366924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:18:23.366924+0000","last_clean_scrub_stamp":"2024-09-18T07:18:23.366924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:09.256470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045339399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'5270","reported_seq":5288,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.274742+0000","last_change":"2024-09-18T07:14:22.288367+0000","last_active":"2024-09-18T07:18:19.274742+0000","last_peered":"2024-09-18T07:18:19.274742+0000","last_clean":"2024-09-18T07:18:19.274742+0000","last_became_active":"2024-09-18T07:14:22.288096+0000","last_became_peered":"2024-09-18T07:14:22.288096+0000","last_unstale":"2024-09-18T07:18:19.274742+0000","last_undegraded":"2024-09-18T07:18:19.274742+0000","last_fullsized":"2024-09-18T07:18:19.274742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5270,"log_dups_size":0,"ondisk_log_size":5270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:39:40.638354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5267,"num_write_kb":201861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'3608","reported_seq":3636,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.236552+0000","last_change":"2024-09-18T07:14:22.284180+0000","last_active":"2024-09-18T07:18:19.236552+0000","last_peered":"2024-09-18T07:18:19.236552+0000","last_clean":"2024-09-18T07:18:19.236552+0000","last_became_active":"2024-09-18T07:14:22.283205+0000","last_became_peered":"2024-09-18T07:14:22.283205+0000","last_unstale":"2024-09-18T07:18:19.236552+0000","last_undegraded":"2024-09-18T07:18:19.236552+0000","last_fullsized":"2024-09-18T07:18:19.236552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3608,"log_dups_size":0,"ondisk_log_size":3608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:36:44.407465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92283408,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":37,"num_read_kb":29,"num_write":3631,"num_write_kb":171797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.274368+0000","last_change":"2024-09-18T07:14:15.255874+0000","last_active":"2024-09-18T07:18:19.274368+0000","last_peered":"2024-09-18T07:18:19.274368+0000","last_clean":"2024-09-18T07:18:19.274368+0000","last_became_active":"2024-09-18T07:14:13.243579+0000","last_became_peered":"2024-09-18T07:14:13.243579+0000","last_unstale":"2024-09-18T07:18:19.274368+0000","last_undegraded":"2024-09-18T07:18:19.274368+0000","last_fullsized":"2024-09-18T07:18:19.274368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:28.722433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096575000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"26'4958","reported_seq":4985,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.281326+0000","last_change":"2024-09-18T07:14:22.475468+0000","last_active":"2024-09-18T07:18:19.281326+0000","last_peered":"2024-09-18T07:18:19.281326+0000","last_clean":"2024-09-18T07:18:19.281326+0000","last_became_active":"2024-09-18T07:14:22.474980+0000","last_became_peered":"2024-09-18T07:14:22.474980+0000","last_unstale":"2024-09-18T07:18:19.281326+0000","last_undegraded":"2024-09-18T07:18:19.281326+0000","last_fullsized":"2024-09-18T07:18:19.281326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4958,"log_dups_size":0,"ondisk_log_size":4958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124452864,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4927,"num_write_kb":170572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:23.737655+0000","last_change":"2024-09-18T07:18:23.737608+0000","last_active":"2024-09-18T07:18:23.737655+0000","last_peered":"2024-09-18T07:18:23.737655+0000","last_clean":"2024-09-18T07:18:23.737655+0000","last_became_active":"2024-09-18T07:14:13.392213+0000","last_became_peered":"2024-09-18T07:14:13.392213+0000","last_unstale":"2024-09-18T07:18:23.737655+0000","last_undegraded":"2024-09-18T07:18:23.737655+0000","last_fullsized":"2024-09-18T07:18:23.737655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:18:23.737546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:18:23.737546+0000","last_clean_scrub_stamp":"2024-09-18T07:18:23.737546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:36:19.791872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000312059,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"26'3082","reported_seq":3120,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:23.526614+0000","last_change":"2024-09-18T07:18:23.526614+0000","last_active":"2024-09-18T07:18:23.526614+0000","last_peered":"2024-09-18T07:18:23.526614+0000","last_clean":"2024-09-18T07:18:23.526614+0000","last_became_active":"2024-09-18T07:14:22.286007+0000","last_became_peered":"2024-09-18T07:14:22.286007+0000","last_unstale":"2024-09-18T07:18:23.526614+0000","last_undegraded":"2024-09-18T07:18:23.526614+0000","last_fullsized":"2024-09-18T07:18:23.526614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3082","last_scrub_stamp":"2024-09-18T07:18:23.526530+0000","last_deep_scrub":"26'3082","last_deep_scrub_stamp":"2024-09-18T07:18:23.526530+0000","last_clean_scrub_stamp":"2024-09-18T07:18:23.526530+0000","objects_scrubbed":24,"log_size":3082,"log_dups_size":0,"ondisk_log_size":3082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:59:59.245914+0000","scrub_duration":169,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92405760,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":3056,"num_write_kb":144258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.246309+0000","last_change":"2024-09-18T07:14:15.603018+0000","last_active":"2024-09-18T07:18:19.246309+0000","last_peered":"2024-09-18T07:18:19.246309+0000","last_clean":"2024-09-18T07:18:19.246309+0000","last_became_active":"2024-09-18T07:14:13.251213+0000","last_became_peered":"2024-09-18T07:14:13.251213+0000","last_unstale":"2024-09-18T07:18:19.246309+0000","last_undegraded":"2024-09-18T07:18:19.246309+0000","last_fullsized":"2024-09-18T07:18:19.246309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:43:22.952115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046971900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"26'5359","reported_seq":5391,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:24.401882+0000","last_change":"2024-09-18T07:18:24.401882+0000","last_active":"2024-09-18T07:18:24.401882+0000","last_peered":"2024-09-18T07:18:24.401882+0000","last_clean":"2024-09-18T07:18:24.401882+0000","last_became_active":"2024-09-18T07:14:22.283928+0000","last_became_peered":"2024-09-18T07:14:22.283928+0000","last_unstale":"2024-09-18T07:18:24.401882+0000","last_undegraded":"2024-09-18T07:18:24.401882+0000","last_fullsized":"2024-09-18T07:18:24.401882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5359","last_scrub_stamp":"2024-09-18T07:18:24.401752+0000","last_deep_scrub":"26'5359","last_deep_scrub_stamp":"2024-09-18T07:18:24.401752+0000","last_clean_scrub_stamp":"2024-09-18T07:18:24.401752+0000","objects_scrubbed":23,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:38:19.981269+0000","scrub_duration":173,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080896,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":18,"num_read_kb":13,"num_write":5351,"num_write_kb":144518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.274383+0000","last_change":"2024-09-18T07:14:15.255301+0000","last_active":"2024-09-18T07:18:19.274383+0000","last_peered":"2024-09-18T07:18:19.274383+0000","last_clean":"2024-09-18T07:18:19.274383+0000","last_became_active":"2024-09-18T07:14:13.243793+0000","last_became_peered":"2024-09-18T07:14:13.243793+0000","last_unstale":"2024-09-18T07:18:19.274383+0000","last_undegraded":"2024-09-18T07:18:19.274383+0000","last_fullsized":"2024-09-18T07:18:19.274383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:12.234148+0000","last_clean_scrub_stamp":"2024-09-18T07:14:12.234148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:24:14.077910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039863000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":123,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.246348+0000","last_change":"2024-09-18T07:14:06.881800+0000","last_active":"2024-09-18T07:18:19.246348+0000","last_peered":"2024-09-18T07:18:19.246348+0000","last_clean":"2024-09-18T07:18:19.246348+0000","last_became_active":"2024-09-18T07:14:06.880771+0000","last_became_peered":"2024-09-18T07:14:06.880771+0000","last_unstale":"2024-09-18T07:18:19.246348+0000","last_undegraded":"2024-09-18T07:18:19.246348+0000","last_fullsized":"2024-09-18T07:18:19.246348+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:05.472923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:05.472923+0000","last_clean_scrub_stamp":"2024-09-18T07:14:05.472923+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:07.064297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"26'5236","reported_seq":5254,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.286888+0000","last_change":"2024-09-18T07:14:22.280027+0000","last_active":"2024-09-18T07:18:19.286888+0000","last_peered":"2024-09-18T07:18:19.286888+0000","last_clean":"2024-09-18T07:18:19.286888+0000","last_became_active":"2024-09-18T07:14:22.279712+0000","last_became_peered":"2024-09-18T07:14:22.279712+0000","last_unstale":"2024-09-18T07:18:19.286888+0000","last_undegraded":"2024-09-18T07:18:19.286888+0000","last_fullsized":"2024-09-18T07:18:19.286888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5236,"log_dups_size":0,"ondisk_log_size":5236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:35:31.397370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136970240,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5162,"num_write_kb":248325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"26'8274","reported_seq":8421,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.303478+0000","last_change":"2024-09-18T07:14:22.475260+0000","last_active":"2024-09-18T07:18:19.303478+0000","last_peered":"2024-09-18T07:18:19.303478+0000","last_clean":"2024-09-18T07:18:19.303478+0000","last_became_active":"2024-09-18T07:14:22.475044+0000","last_became_peered":"2024-09-18T07:14:22.475044+0000","last_unstale":"2024-09-18T07:18:19.303478+0000","last_undegraded":"2024-09-18T07:18:19.303478+0000","last_fullsized":"2024-09-18T07:18:19.303478+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":8274,"log_dups_size":0,"ondisk_log_size":8274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:52:46.307809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":409,"num_read_kb":432,"num_write":5209,"num_write_kb":145235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"26'6939","reported_seq":6959,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.276199+0000","last_change":"2024-09-18T07:14:22.286302+0000","last_active":"2024-09-18T07:18:19.276199+0000","last_peered":"2024-09-18T07:18:19.276199+0000","last_clean":"2024-09-18T07:18:19.276199+0000","last_became_active":"2024-09-18T07:14:22.285050+0000","last_became_peered":"2024-09-18T07:14:22.285050+0000","last_unstale":"2024-09-18T07:18:19.276199+0000","last_undegraded":"2024-09-18T07:18:19.276199+0000","last_fullsized":"2024-09-18T07:18:19.276199+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":6939,"log_dups_size":0,"ondisk_log_size":6939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:36:34.664394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146120704,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":6915,"num_write_kb":222193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"24'4813","reported_seq":4847,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.271499+0000","last_change":"2024-09-18T07:14:22.284686+0000","last_active":"2024-09-18T07:18:19.271499+0000","last_peered":"2024-09-18T07:18:19.271499+0000","last_clean":"2024-09-18T07:18:19.271499+0000","last_became_active":"2024-09-18T07:14:22.284397+0000","last_became_peered":"2024-09-18T07:14:22.284397+0000","last_unstale":"2024-09-18T07:18:19.271499+0000","last_undegraded":"2024-09-18T07:18:19.271499+0000","last_fullsized":"2024-09-18T07:18:19.271499+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4813,"log_dups_size":0,"ondisk_log_size":4813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:02:38.728384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104988672,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":64,"num_read_kb":50,"num_write":4839,"num_write_kb":163526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"26'4926","reported_seq":4955,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.285422+0000","last_change":"2024-09-18T07:14:22.288110+0000","last_active":"2024-09-18T07:18:19.285422+0000","last_peered":"2024-09-18T07:18:19.285422+0000","last_clean":"2024-09-18T07:18:19.285422+0000","last_became_active":"2024-09-18T07:14:22.287513+0000","last_became_peered":"2024-09-18T07:14:22.287513+0000","last_unstale":"2024-09-18T07:18:19.285422+0000","last_undegraded":"2024-09-18T07:18:19.285422+0000","last_fullsized":"2024-09-18T07:18:19.285422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:02:43.799881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794368,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":43,"num_read_kb":96,"num_write":4998,"num_write_kb":181316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"26'3048","reported_seq":3068,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.246538+0000","last_change":"2024-09-18T07:14:22.289036+0000","last_active":"2024-09-18T07:18:19.246538+0000","last_peered":"2024-09-18T07:18:19.246538+0000","last_clean":"2024-09-18T07:18:19.246538+0000","last_became_active":"2024-09-18T07:14:22.288690+0000","last_became_peered":"2024-09-18T07:14:22.288690+0000","last_unstale":"2024-09-18T07:18:19.246538+0000","last_undegraded":"2024-09-18T07:18:19.246538+0000","last_fullsized":"2024-09-18T07:18:19.246538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3048,"log_dups_size":0,"ondisk_log_size":3048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:02:01.547710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92275200,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":3030,"num_write_kb":128003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"26'3896","reported_seq":4006,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.290232+0000","last_change":"2024-09-18T07:14:22.289068+0000","last_active":"2024-09-18T07:18:19.290232+0000","last_peered":"2024-09-18T07:18:19.290232+0000","last_clean":"2024-09-18T07:18:19.290232+0000","last_became_active":"2024-09-18T07:14:22.288767+0000","last_became_peered":"2024-09-18T07:14:22.288767+0000","last_unstale":"2024-09-18T07:18:19.290232+0000","last_undegraded":"2024-09-18T07:18:19.290232+0000","last_fullsized":"2024-09-18T07:18:19.290232+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3896,"log_dups_size":0,"ondisk_log_size":3896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:08:25.598857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106366632,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":300,"num_read_kb":235,"num_write":4144,"num_write_kb":154845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"26'5120","reported_seq":5138,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.282387+0000","last_change":"2024-09-18T07:14:22.289787+0000","last_active":"2024-09-18T07:18:19.282387+0000","last_peered":"2024-09-18T07:18:19.282387+0000","last_clean":"2024-09-18T07:18:19.282387+0000","last_became_active":"2024-09-18T07:14:22.289527+0000","last_became_peered":"2024-09-18T07:14:22.289527+0000","last_unstale":"2024-09-18T07:18:19.282387+0000","last_undegraded":"2024-09-18T07:18:19.282387+0000","last_fullsized":"2024-09-18T07:18:19.282387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":5120,"log_dups_size":0,"ondisk_log_size":5120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:39:43.412358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130834432,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5068,"num_write_kb":190489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"26'3029","reported_seq":3352,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.284191+0000","last_change":"2024-09-18T07:14:22.285350+0000","last_active":"2024-09-18T07:18:19.284191+0000","last_peered":"2024-09-18T07:18:19.284191+0000","last_clean":"2024-09-18T07:18:19.284191+0000","last_became_active":"2024-09-18T07:14:22.284989+0000","last_became_peered":"2024-09-18T07:14:22.284989+0000","last_unstale":"2024-09-18T07:18:19.284191+0000","last_undegraded":"2024-09-18T07:18:19.284191+0000","last_fullsized":"2024-09-18T07:18:19.284191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3029,"log_dups_size":0,"ondisk_log_size":3029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:58:31.394769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71434240,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1022,"num_read_kb":916,"num_write":3192,"num_write_kb":144514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"24'3065","reported_seq":3106,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.246340+0000","last_change":"2024-09-18T07:14:22.290602+0000","last_active":"2024-09-18T07:18:19.246340+0000","last_peered":"2024-09-18T07:18:19.246340+0000","last_clean":"2024-09-18T07:18:19.246340+0000","last_became_active":"2024-09-18T07:14:22.290299+0000","last_became_peered":"2024-09-18T07:14:22.290299+0000","last_unstale":"2024-09-18T07:18:19.246340+0000","last_undegraded":"2024-09-18T07:18:19.246340+0000","last_fullsized":"2024-09-18T07:18:19.246340+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3065,"log_dups_size":0,"ondisk_log_size":3065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:31:47.964089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":3095,"num_write_kb":160450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"26'3707","reported_seq":3725,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.282362+0000","last_change":"2024-09-18T07:14:22.290568+0000","last_active":"2024-09-18T07:18:19.282362+0000","last_peered":"2024-09-18T07:18:19.282362+0000","last_clean":"2024-09-18T07:18:19.282362+0000","last_became_active":"2024-09-18T07:14:22.290308+0000","last_became_peered":"2024-09-18T07:14:22.290308+0000","last_unstale":"2024-09-18T07:18:19.282362+0000","last_undegraded":"2024-09-18T07:18:19.282362+0000","last_fullsized":"2024-09-18T07:18:19.282362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":3707,"log_dups_size":0,"ondisk_log_size":3707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:17.589485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120979456,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3652,"num_write_kb":176387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"26'7745","reported_seq":7774,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:19.246450+0000","last_change":"2024-09-18T07:14:22.288236+0000","last_active":"2024-09-18T07:18:19.246450+0000","last_peered":"2024-09-18T07:18:19.246450+0000","last_clean":"2024-09-18T07:18:19.246450+0000","last_became_active":"2024-09-18T07:14:22.287927+0000","last_became_peered":"2024-09-18T07:14:22.287927+0000","last_unstale":"2024-09-18T07:18:19.246450+0000","last_undegraded":"2024-09-18T07:18:19.246450+0000","last_fullsized":"2024-09-18T07:18:19.246450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:14:21.266057+0000","last_clean_scrub_stamp":"2024-09-18T07:14:21.266057+0000","objects_scrubbed":0,"log_size":7745,"log_dups_size":0,"ondisk_log_size":7745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:26:51.355433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":31,"num_read_kb":87,"num_write":7741,"num_write_kb":165450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3581698296,"num_objects":881,"num_object_clones":0,"num_object_copies":1762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":0,"num_read":2575,"num_read_kb":2591,"num_write":165342,"num_write_kb":5659041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6865731584,"data_stored":6861464696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":168288,"ondisk_log_size":168288,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1309720,"kb_used_data":1043972,"kb_used_omap":1,"kb_used_meta":265726,"kb_avail":93062120,"statfs":{"total":96636764160,"available":95295610880,"internally_reserved":0,"allocated":1069027328,"data_stored":1068302773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":272103882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408848,"kb_used_data":311036,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93962992,"statfs":{"total":96636764160,"available":96218103808,"internally_reserved":0,"allocated":318500864,"data_stored":318134177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":100137414},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542198,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1674096,"kb_used_data":1300128,"kb_used_omap":1,"kb_used_meta":373950,"kb_avail":92697744,"statfs":{"total":96636764160,"available":94922489856,"internally_reserved":0,"allocated":1331331072,"data_stored":1330392625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":382925253},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542198,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1384288,"kb_used_data":1082444,"kb_used_omap":1,"kb_used_meta":301822,"kb_avail":92987552,"statfs":{"total":96636764160,"available":95219253248,"internally_reserved":0,"allocated":1108422656,"data_stored":1107555517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":309066183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436756,"kb_used_data":338944,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93935084,"statfs":{"total":96636764160,"available":96189526016,"internally_reserved":0,"allocated":347078656,"data_stored":346704641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":100137413},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542198,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902312,"kb_used_data":676824,"kb_used_omap":1,"kb_used_meta":225470,"kb_avail":93469528,"statfs":{"total":96636764160,"available":95712796672,"internally_reserved":0,"allocated":693067776,"data_stored":692416132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":230881733},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542198,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1276100,"kb_used_data":1002036,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":93095740,"statfs":{"total":96636764160,"available":95330037760,"internally_reserved":0,"allocated":1026084864,"data_stored":1025252949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":280623557},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1182052,"kb_used_data":952400,"kb_used_omap":1,"kb_used_meta":229630,"kb_avail":93189788,"statfs":{"total":96636764160,"available":95426342912,"internally_reserved":0,"allocated":975257600,"data_stored":974458280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235141573},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1025306624,"data_stored":1024637208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":692817920,"data_stored":692329300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346832896,"data_stored":346617828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1108176896,"data_stored":1107468704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1331085312,"data_stored":1330305812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":975007744,"data_stored":974371448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":318255104,"data_stored":318047364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1068249088,"data_stored":1067687032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T07:18:26.013 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.013 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.013 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.013 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.013 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.014 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.014 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.014 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.014 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.014 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.014 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.014 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.015 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-18T07:14:12.234148+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=12, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.015 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.015 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-18T07:14:12.234148+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=12, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.015 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.015 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.015 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.015 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-18T07:14:12.234148+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=12, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.016 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.016 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-18T07:14:12.234148+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=12, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.016 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-18T07:14:12.234148+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=12, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.016 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-18T07:14:05.472923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=5, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.016 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.016 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.016 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.016 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.017 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.017 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.017 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.017 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.017 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.017 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.017 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.018 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-18T07:14:21.266057+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=14, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=7, tm_min=18, tm_sec=21, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T07:18:26.018 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-18T07:18:46.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T07:18:46.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.126+0000 7efdebb64640 1 Processor -- start 2024-09-18T07:18:46.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.127+0000 7efdebb64640 1 -- start start 2024-09-18T07:18:46.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.127+0000 7efdebb64640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde40aae80 con 0x7efde413f9f0 2024-09-18T07:18:46.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.127+0000 7efdebb64640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde40731d0 con 0x7efde4143ec0 2024-09-18T07:18:46.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.127+0000 7efdebb64640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde4140f40 con 0x7efde4143ae0 2024-09-18T07:18:46.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.127+0000 7efde9900640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7efde4143ae0 0x7efde4072ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:54070/0 (socket says 172.21.15.70:54070) 2024-09-18T07:18:46.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.127+0000 7efde9900640 1 -- 172.21.15.70:0/1757052295 learned_addr learned my addr 172.21.15.70:0/1757052295 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:46.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.128+0000 7efde88fe640 1 -- 172.21.15.70:0/1757052295 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 896489223 0 0) 0x7efde4140f40 con 0x7efde4143ae0 2024-09-18T07:18:46.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.128+0000 7efde88fe640 1 -- 172.21.15.70:0/1757052295 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdd0003520 con 0x7efde4143ae0 2024-09-18T07:18:46.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.128+0000 7efde88fe640 1 -- 172.21.15.70:0/1757052295 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3556411354 0 0) 0x7efde40aae80 con 0x7efde413f9f0 2024-09-18T07:18:46.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.128+0000 7efde88fe640 1 -- 172.21.15.70:0/1757052295 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efde4140f40 con 0x7efde413f9f0 2024-09-18T07:18:46.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.128+0000 7efde88fe640 1 -- 172.21.15.70:0/1757052295 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1373477374 0 0) 0x7efdd0003520 con 0x7efde4143ae0 2024-09-18T07:18:46.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.128+0000 7efde88fe640 1 -- 172.21.15.70:0/1757052295 >> v1:172.21.15.172:6789/0 conn(0x7efde4143ec0 legacy=0x7efde413f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.128+0000 7efde88fe640 1 -- 172.21.15.70:0/1757052295 >> v1:172.21.15.70:6789/0 conn(0x7efde413f9f0 legacy=0x7efde4070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.128+0000 7efde88fe640 1 -- 172.21.15.70:0/1757052295 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde4073f80 con 0x7efde4143ae0 2024-09-18T07:18:46.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.128+0000 7efde88fe640 1 -- 172.21.15.70:0/1757052295 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7efdd4001690 con 0x7efde4143ae0 2024-09-18T07:18:46.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.128+0000 7efde88fe640 1 -- 172.21.15.70:0/1757052295 <== mon.2 v1:172.21.15.70:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efdd40018b0 con 0x7efde4143ae0 2024-09-18T07:18:46.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.128+0000 7efde88fe640 1 -- 172.21.15.70:0/1757052295 <== mon.2 v1:172.21.15.70:6790/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7efdd4001d40 con 0x7efde4143ae0 2024-09-18T07:18:46.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.129+0000 7efdebb64640 1 -- 172.21.15.70:0/1757052295 >> v1:172.21.15.70:6790/0 conn(0x7efde4143ae0 legacy=0x7efde4072ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.129+0000 7efdebb64640 1 -- 172.21.15.70:0/1757052295 shutdown_connections 2024-09-18T07:18:46.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.129+0000 7efdebb64640 1 -- 172.21.15.70:0/1757052295 >> 172.21.15.70:0/1757052295 conn(0x7efde40b5790 msgr2=0x7efde40b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:46.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.129+0000 7efdebb64640 1 -- 172.21.15.70:0/1757052295 shutdown_connections 2024-09-18T07:18:46.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.129+0000 7efdebb64640 1 -- 172.21.15.70:0/1757052295 wait complete. 2024-09-18T07:18:46.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.130+0000 7efdebb64640 1 Processor -- start 2024-09-18T07:18:46.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.130+0000 7efdebb64640 1 -- start start 2024-09-18T07:18:46.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.130+0000 7efdebb64640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde41d9a20 con 0x7efde4143ec0 2024-09-18T07:18:46.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.130+0000 7efdebb64640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde41cc190 con 0x7efde413f9f0 2024-09-18T07:18:46.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.130+0000 7efdebb64640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde413a930 con 0x7efde4143ae0 2024-09-18T07:18:46.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.130+0000 7efde90ff640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7efde4143ae0 0x7efde4139c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:54078/0 (socket says 172.21.15.70:54078) 2024-09-18T07:18:46.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.130+0000 7efde90ff640 1 -- 172.21.15.70:0/2074011638 learned_addr learned my addr 172.21.15.70:0/2074011638 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:46.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1880862025 0 0) 0x7efde41d9a20 con 0x7efde4143ec0 2024-09-18T07:18:46.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdc8003520 con 0x7efde4143ec0 2024-09-18T07:18:46.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 229375713 0 0) 0x7efde413a930 con 0x7efde4143ae0 2024-09-18T07:18:46.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efde41d9a20 con 0x7efde4143ae0 2024-09-18T07:18:46.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 474440384 0 0) 0x7efde41cc190 con 0x7efde413f9f0 2024-09-18T07:18:46.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efde413a930 con 0x7efde413f9f0 2024-09-18T07:18:46.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3187403295 0 0) 0x7efdc8003520 con 0x7efde4143ec0 2024-09-18T07:18:46.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efde41cc190 con 0x7efde4143ec0 2024-09-18T07:18:46.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7efddc001770 con 0x7efde4143ec0 2024-09-18T07:18:46.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2881279914 0 0) 0x7efde41d9a20 con 0x7efde4143ae0 2024-09-18T07:18:46.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efdc8003520 con 0x7efde4143ae0 2024-09-18T07:18:46.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7efde0001c30 con 0x7efde4143ae0 2024-09-18T07:18:46.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2367494265 0 0) 0x7efde413a930 con 0x7efde413f9f0 2024-09-18T07:18:46.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 --> v1:172.21.15.172:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efde41d9a20 con 0x7efde413f9f0 2024-09-18T07:18:46.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7efdd4001a60 con 0x7efde413f9f0 2024-09-18T07:18:46.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2517923103 0 0) 0x7efde41cc190 con 0x7efde4143ec0 2024-09-18T07:18:46.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 >> v1:172.21.15.70:6790/0 conn(0x7efde4143ae0 legacy=0x7efde4139c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 >> v1:172.21.15.172:6789/0 conn(0x7efde413f9f0 legacy=0x7efde41dc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde413ab00 con 0x7efde4143ec0 2024-09-18T07:18:46.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdebb64640 1 -- 172.21.15.70:0/2074011638 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7efde413a340 con 0x7efde4143ec0 2024-09-18T07:18:46.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdebb64640 1 -- 172.21.15.70:0/2074011638 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7efde41cafd0 con 0x7efde4143ec0 2024-09-18T07:18:46.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efddc001d20 con 0x7efde4143ec0 2024-09-18T07:18:46.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.131+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7efddc003790 con 0x7efde4143ec0 2024-09-18T07:18:46.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.132+0000 7efdebb64640 1 -- 172.21.15.70:0/2074011638 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdcc005080 con 0x7efde4143ec0 2024-09-18T07:18:46.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.133+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7efddc01c3c0 con 0x7efde4143ec0 2024-09-18T07:18:46.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.134+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7efddc094ba0 con 0x7efde4143ec0 2024-09-18T07:18:46.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.138+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7efddc05d4e0 con 0x7efde4143ec0 2024-09-18T07:18:46.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.311+0000 7efdebb64640 1 -- 172.21.15.70:0/2074011638 --> v1:172.21.15.70:6816/1657990791 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efdcc000b90 con 0x7efdc8079900 2024-09-18T07:18:46.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.313+0000 7efdda7fc640 1 -- 172.21.15.70:0/2074011638 <== mgr.4104 v1:172.21.15.70:6816/1657990791 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117796 (unknown 2965378022 0 2130871872) 0x7efdcc000b90 con 0x7efdc8079900 2024-09-18T07:18:46.317 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-18T07:18:46.317 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-18T07:18:46.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.315+0000 7efdebb64640 1 -- 172.21.15.70:0/2074011638 >> v1:172.21.15.70:6816/1657990791 conn(0x7efdc8079900 legacy=0x7efdc807bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.315+0000 7efdebb64640 1 -- 172.21.15.70:0/2074011638 >> v1:172.21.15.70:6789/0 conn(0x7efde4143ec0 legacy=0x7efde41ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.315+0000 7efdebb64640 1 -- 172.21.15.70:0/2074011638 shutdown_connections 2024-09-18T07:18:46.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.315+0000 7efdebb64640 1 -- 172.21.15.70:0/2074011638 >> 172.21.15.70:0/2074011638 conn(0x7efde40b5790 msgr2=0x7efde413b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:46.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.315+0000 7efdebb64640 1 -- 172.21.15.70:0/2074011638 shutdown_connections 2024-09-18T07:18:46.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.315+0000 7efdebb64640 1 -- 172.21.15.70:0/2074011638 wait complete. 2024-09-18T07:18:46.328 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-18T07:18:45.501425+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3582288683,"num_objects":885,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":2641,"num_read_kb":2646,"num_write":165445,"num_write_kb":5660671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":169164,"ondisk_log_size":169164,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8582472,"kb_used_data":6707784,"kb_used_omap":12,"kb_used_meta":1874547,"kb_avail":746392248,"statfs":{"total":773094113280,"available":764305661952,"internally_reserved":0,"allocated":6868770816,"data_stored":6863217094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1919536688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002451"},"pg_stats":[{"pgid":"3.1f","version":"26'3641","reported_seq":3671,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:28.491805+0000","last_change":"2024-09-18T07:18:28.491719+0000","last_active":"2024-09-18T07:18:28.491805+0000","last_peered":"2024-09-18T07:18:28.491805+0000","last_clean":"2024-09-18T07:18:28.491805+0000","last_became_active":"2024-09-18T07:14:22.286401+0000","last_became_peered":"2024-09-18T07:14:22.286401+0000","last_unstale":"2024-09-18T07:18:28.491805+0000","last_undegraded":"2024-09-18T07:18:28.491805+0000","last_fullsized":"2024-09-18T07:18:28.491805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3641","last_scrub_stamp":"2024-09-18T07:18:28.491609+0000","last_deep_scrub":"26'3641","last_deep_scrub_stamp":"2024-09-18T07:18:28.491609+0000","last_clean_scrub_stamp":"2024-09-18T07:18:28.491609+0000","objects_scrubbed":24,"log_size":3641,"log_dups_size":0,"ondisk_log_size":3641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:00:04.463081+0000","scrub_duration":192,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95160672,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":3623,"num_write_kb":151525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"26'6443","reported_seq":6572,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:29.568250+0000","last_change":"2024-09-18T07:18:29.568250+0000","last_active":"2024-09-18T07:18:29.568250+0000","last_peered":"2024-09-18T07:18:29.568250+0000","last_clean":"2024-09-18T07:18:29.568250+0000","last_became_active":"2024-09-18T07:14:22.284487+0000","last_became_peered":"2024-09-18T07:14:22.284487+0000","last_unstale":"2024-09-18T07:18:29.568250+0000","last_undegraded":"2024-09-18T07:18:29.568250+0000","last_fullsized":"2024-09-18T07:18:29.568250+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6443","last_scrub_stamp":"2024-09-18T07:18:29.568139+0000","last_deep_scrub":"26'6443","last_deep_scrub_stamp":"2024-09-18T07:18:29.568139+0000","last_clean_scrub_stamp":"2024-09-18T07:18:29.568139+0000","objects_scrubbed":28,"log_size":6443,"log_dups_size":0,"ondisk_log_size":6443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:18:22.326064+0000","scrub_duration":248,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377808,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":6397,"num_write_kb":190429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"26'7629","reported_seq":7690,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:31.239947+0000","last_change":"2024-09-18T07:18:31.239947+0000","last_active":"2024-09-18T07:18:31.239947+0000","last_peered":"2024-09-18T07:18:31.239947+0000","last_clean":"2024-09-18T07:18:31.239947+0000","last_became_active":"2024-09-18T07:14:22.288441+0000","last_became_peered":"2024-09-18T07:14:22.288441+0000","last_unstale":"2024-09-18T07:18:31.239947+0000","last_undegraded":"2024-09-18T07:18:31.239947+0000","last_fullsized":"2024-09-18T07:18:31.239947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'7629","last_scrub_stamp":"2024-09-18T07:18:31.239855+0000","last_deep_scrub":"26'7629","last_deep_scrub_stamp":"2024-09-18T07:18:31.239855+0000","last_clean_scrub_stamp":"2024-09-18T07:18:31.239855+0000","objects_scrubbed":41,"log_size":7629,"log_dups_size":0,"ondisk_log_size":7629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:19:59.168874+0000","scrub_duration":401,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167903232,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":127,"num_read_kb":103,"num_write":7620,"num_write_kb":243849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"26'3278","reported_seq":3324,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:28.953914+0000","last_change":"2024-09-18T07:18:28.953914+0000","last_active":"2024-09-18T07:18:28.953914+0000","last_peered":"2024-09-18T07:18:28.953914+0000","last_clean":"2024-09-18T07:18:28.953914+0000","last_became_active":"2024-09-18T07:14:22.287853+0000","last_became_peered":"2024-09-18T07:14:22.287853+0000","last_unstale":"2024-09-18T07:18:28.953914+0000","last_undegraded":"2024-09-18T07:18:28.953914+0000","last_fullsized":"2024-09-18T07:18:28.953914+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3278","last_scrub_stamp":"2024-09-18T07:18:28.953843+0000","last_deep_scrub":"26'3278","last_deep_scrub_stamp":"2024-09-18T07:18:28.953843+0000","last_clean_scrub_stamp":"2024-09-18T07:18:28.953843+0000","objects_scrubbed":23,"log_size":3278,"log_dups_size":0,"ondisk_log_size":3278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:57:39.262111+0000","scrub_duration":180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88211456,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":79,"num_read_kb":61,"num_write":3296,"num_write_kb":156739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"26'3854","reported_seq":3884,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:27.429647+0000","last_change":"2024-09-18T07:18:27.429647+0000","last_active":"2024-09-18T07:18:27.429647+0000","last_peered":"2024-09-18T07:18:27.429647+0000","last_clean":"2024-09-18T07:18:27.429647+0000","last_became_active":"2024-09-18T07:14:22.286018+0000","last_became_peered":"2024-09-18T07:14:22.286018+0000","last_unstale":"2024-09-18T07:18:27.429647+0000","last_undegraded":"2024-09-18T07:18:27.429647+0000","last_fullsized":"2024-09-18T07:18:27.429647+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3854","last_scrub_stamp":"2024-09-18T07:18:27.429579+0000","last_deep_scrub":"26'3854","last_deep_scrub_stamp":"2024-09-18T07:18:27.429579+0000","last_clean_scrub_stamp":"2024-09-18T07:18:27.429579+0000","objects_scrubbed":24,"log_size":3854,"log_dups_size":0,"ondisk_log_size":3854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:16:43.411527+0000","scrub_duration":160,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91650400,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":3788,"num_write_kb":184441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"26'5592","reported_seq":5623,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:28.106599+0000","last_change":"2024-09-18T07:18:28.106599+0000","last_active":"2024-09-18T07:18:28.106599+0000","last_peered":"2024-09-18T07:18:28.106599+0000","last_clean":"2024-09-18T07:18:28.106599+0000","last_became_active":"2024-09-18T07:14:22.279826+0000","last_became_peered":"2024-09-18T07:14:22.279826+0000","last_unstale":"2024-09-18T07:18:28.106599+0000","last_undegraded":"2024-09-18T07:18:28.106599+0000","last_fullsized":"2024-09-18T07:18:28.106599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5592","last_scrub_stamp":"2024-09-18T07:18:28.106504+0000","last_deep_scrub":"26'5592","last_deep_scrub_stamp":"2024-09-18T07:18:28.106504+0000","last_clean_scrub_stamp":"2024-09-18T07:18:28.106504+0000","objects_scrubbed":26,"log_size":5592,"log_dups_size":0,"ondisk_log_size":5592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:40:23.902477+0000","scrub_duration":215,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":15,"num_read_kb":75,"num_write":5550,"num_write_kb":155179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"26'4770","reported_seq":4813,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:27.600712+0000","last_change":"2024-09-18T07:18:27.600712+0000","last_active":"2024-09-18T07:18:27.600712+0000","last_peered":"2024-09-18T07:18:27.600712+0000","last_clean":"2024-09-18T07:18:27.600712+0000","last_became_active":"2024-09-18T07:14:22.279885+0000","last_became_peered":"2024-09-18T07:14:22.279885+0000","last_unstale":"2024-09-18T07:18:27.600712+0000","last_undegraded":"2024-09-18T07:18:27.600712+0000","last_fullsized":"2024-09-18T07:18:27.600712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4770","last_scrub_stamp":"2024-09-18T07:18:27.600630+0000","last_deep_scrub":"26'4770","last_deep_scrub_stamp":"2024-09-18T07:18:27.600630+0000","last_clean_scrub_stamp":"2024-09-18T07:18:27.600630+0000","objects_scrubbed":25,"log_size":4770,"log_dups_size":0,"ondisk_log_size":4770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:48:54.518510+0000","scrub_duration":131,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":4713,"num_write_kb":147181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"26'8038","reported_seq":8067,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:28.538516+0000","last_change":"2024-09-18T07:18:28.538516+0000","last_active":"2024-09-18T07:18:28.538516+0000","last_peered":"2024-09-18T07:18:28.538516+0000","last_clean":"2024-09-18T07:18:28.538516+0000","last_became_active":"2024-09-18T07:14:22.283231+0000","last_became_peered":"2024-09-18T07:14:22.283231+0000","last_unstale":"2024-09-18T07:18:28.538516+0000","last_undegraded":"2024-09-18T07:18:28.538516+0000","last_fullsized":"2024-09-18T07:18:28.538516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'8038","last_scrub_stamp":"2024-09-18T07:18:28.538455+0000","last_deep_scrub":"26'8038","last_deep_scrub_stamp":"2024-09-18T07:18:28.538455+0000","last_clean_scrub_stamp":"2024-09-18T07:18:28.538455+0000","objects_scrubbed":31,"log_size":8038,"log_dups_size":0,"ondisk_log_size":8038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:28:56.656721+0000","scrub_duration":227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125895176,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":8013,"num_write_kb":214525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"26'7268","reported_seq":7300,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:26.623136+0000","last_change":"2024-09-18T07:18:26.623136+0000","last_active":"2024-09-18T07:18:26.623136+0000","last_peered":"2024-09-18T07:18:26.623136+0000","last_clean":"2024-09-18T07:18:26.623136+0000","last_became_active":"2024-09-18T07:14:22.285884+0000","last_became_peered":"2024-09-18T07:14:22.285884+0000","last_unstale":"2024-09-18T07:18:26.623136+0000","last_undegraded":"2024-09-18T07:18:26.623136+0000","last_fullsized":"2024-09-18T07:18:26.623136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'7268","last_scrub_stamp":"2024-09-18T07:18:26.623058+0000","last_deep_scrub":"26'7268","last_deep_scrub_stamp":"2024-09-18T07:18:26.623058+0000","last_clean_scrub_stamp":"2024-09-18T07:18:26.623058+0000","objects_scrubbed":33,"log_size":7268,"log_dups_size":0,"ondisk_log_size":7268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:11:52.835452+0000","scrub_duration":326,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135659520,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":7238,"num_write_kb":200827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"26'3438","reported_seq":3473,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:30.038962+0000","last_change":"2024-09-18T07:18:30.038962+0000","last_active":"2024-09-18T07:18:30.038962+0000","last_peered":"2024-09-18T07:18:30.038962+0000","last_clean":"2024-09-18T07:18:30.038962+0000","last_became_active":"2024-09-18T07:14:22.289372+0000","last_became_peered":"2024-09-18T07:14:22.289372+0000","last_unstale":"2024-09-18T07:18:30.038962+0000","last_undegraded":"2024-09-18T07:18:30.038962+0000","last_fullsized":"2024-09-18T07:18:30.038962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3438","last_scrub_stamp":"2024-09-18T07:18:30.038847+0000","last_deep_scrub":"26'3438","last_deep_scrub_stamp":"2024-09-18T07:18:30.038847+0000","last_clean_scrub_stamp":"2024-09-18T07:18:30.038847+0000","objects_scrubbed":29,"log_size":3438,"log_dups_size":0,"ondisk_log_size":3438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:26:17.810197+0000","scrub_duration":241,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112003944,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":22,"num_read_kb":143,"num_write":3412,"num_write_kb":151531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"26'8055","reported_seq":8086,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:32.599333+0000","last_change":"2024-09-18T07:18:32.599333+0000","last_active":"2024-09-18T07:18:32.599333+0000","last_peered":"2024-09-18T07:18:32.599333+0000","last_clean":"2024-09-18T07:18:32.599333+0000","last_became_active":"2024-09-18T07:14:22.288634+0000","last_became_peered":"2024-09-18T07:14:22.288634+0000","last_unstale":"2024-09-18T07:18:32.599333+0000","last_undegraded":"2024-09-18T07:18:32.599333+0000","last_fullsized":"2024-09-18T07:18:32.599333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'8055","last_scrub_stamp":"2024-09-18T07:18:32.599266+0000","last_deep_scrub":"26'8055","last_deep_scrub_stamp":"2024-09-18T07:18:32.599266+0000","last_clean_scrub_stamp":"2024-09-18T07:18:32.599266+0000","objects_scrubbed":30,"log_size":8055,"log_dups_size":0,"ondisk_log_size":8055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:09.142729+0000","scrub_duration":258,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8027,"num_write_kb":201731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"26'4892","reported_seq":4920,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:26.050903+0000","last_change":"2024-09-18T07:18:26.050903+0000","last_active":"2024-09-18T07:18:26.050903+0000","last_peered":"2024-09-18T07:18:26.050903+0000","last_clean":"2024-09-18T07:18:26.050903+0000","last_became_active":"2024-09-18T07:14:22.291551+0000","last_became_peered":"2024-09-18T07:14:22.291551+0000","last_unstale":"2024-09-18T07:18:26.050903+0000","last_undegraded":"2024-09-18T07:18:26.050903+0000","last_fullsized":"2024-09-18T07:18:26.050903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4892","last_scrub_stamp":"2024-09-18T07:18:26.050790+0000","last_deep_scrub":"26'4892","last_deep_scrub_stamp":"2024-09-18T07:18:26.050790+0000","last_clean_scrub_stamp":"2024-09-18T07:18:26.050790+0000","objects_scrubbed":18,"log_size":4892,"log_dups_size":0,"ondisk_log_size":4892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:33:05.547339+0000","scrub_duration":181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303680,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4826,"num_write_kb":160133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:27.045339+0000","last_change":"2024-09-18T07:18:27.045294+0000","last_active":"2024-09-18T07:18:27.045339+0000","last_peered":"2024-09-18T07:18:27.045339+0000","last_clean":"2024-09-18T07:18:27.045339+0000","last_became_active":"2024-09-18T07:14:13.251609+0000","last_became_peered":"2024-09-18T07:14:13.251609+0000","last_unstale":"2024-09-18T07:18:27.045339+0000","last_undegraded":"2024-09-18T07:18:27.045339+0000","last_fullsized":"2024-09-18T07:18:27.045339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:18:27.045228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:18:27.045228+0000","last_clean_scrub_stamp":"2024-09-18T07:18:27.045228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:00:38.453213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040903899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'7912","reported_seq":7941,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:24.631713+0000","last_change":"2024-09-18T07:18:24.631667+0000","last_active":"2024-09-18T07:18:24.631713+0000","last_peered":"2024-09-18T07:18:24.631713+0000","last_clean":"2024-09-18T07:18:24.631713+0000","last_became_active":"2024-09-18T07:14:22.282926+0000","last_became_peered":"2024-09-18T07:14:22.282926+0000","last_unstale":"2024-09-18T07:18:24.631713+0000","last_undegraded":"2024-09-18T07:18:24.631713+0000","last_fullsized":"2024-09-18T07:18:24.631713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'7912","last_scrub_stamp":"2024-09-18T07:18:24.631601+0000","last_deep_scrub":"26'7912","last_deep_scrub_stamp":"2024-09-18T07:18:24.631601+0000","last_clean_scrub_stamp":"2024-09-18T07:18:24.631601+0000","objects_scrubbed":34,"log_size":7912,"log_dups_size":0,"ondisk_log_size":7912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:29:00.658275+0000","scrub_duration":252,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141926400,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7854,"num_write_kb":222187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:24.411930+0000","last_change":"2024-09-18T07:18:24.411880+0000","last_active":"2024-09-18T07:18:24.411930+0000","last_peered":"2024-09-18T07:18:24.411930+0000","last_clean":"2024-09-18T07:18:24.411930+0000","last_became_active":"2024-09-18T07:14:13.251585+0000","last_became_peered":"2024-09-18T07:14:13.251585+0000","last_unstale":"2024-09-18T07:18:24.411930+0000","last_undegraded":"2024-09-18T07:18:24.411930+0000","last_fullsized":"2024-09-18T07:18:24.411930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:18:24.411805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:18:24.411805+0000","last_clean_scrub_stamp":"2024-09-18T07:18:24.411805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:55.857411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039885799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"26'4256","reported_seq":4295,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:25.525834+0000","last_change":"2024-09-18T07:18:25.525755+0000","last_active":"2024-09-18T07:18:25.525834+0000","last_peered":"2024-09-18T07:18:25.525834+0000","last_clean":"2024-09-18T07:18:25.525834+0000","last_became_active":"2024-09-18T07:14:22.287843+0000","last_became_peered":"2024-09-18T07:14:22.287843+0000","last_unstale":"2024-09-18T07:18:25.525834+0000","last_undegraded":"2024-09-18T07:18:25.525834+0000","last_fullsized":"2024-09-18T07:18:25.525834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4256","last_scrub_stamp":"2024-09-18T07:18:25.525633+0000","last_deep_scrub":"26'4256","last_deep_scrub_stamp":"2024-09-18T07:18:25.525633+0000","last_clean_scrub_stamp":"2024-09-18T07:18:25.525633+0000","objects_scrubbed":34,"log_size":4256,"log_dups_size":0,"ondisk_log_size":4256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:26:16.144522+0000","scrub_duration":294,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141164544,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":4202,"num_write_kb":175065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:26.438594+0000","last_change":"2024-09-18T07:18:26.438594+0000","last_active":"2024-09-18T07:18:26.438594+0000","last_peered":"2024-09-18T07:18:26.438594+0000","last_clean":"2024-09-18T07:18:26.438594+0000","last_became_active":"2024-09-18T07:14:13.957168+0000","last_became_peered":"2024-09-18T07:14:13.957168+0000","last_unstale":"2024-09-18T07:18:26.438594+0000","last_undegraded":"2024-09-18T07:18:26.438594+0000","last_fullsized":"2024-09-18T07:18:26.438594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:18:26.438533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:18:26.438533+0000","last_clean_scrub_stamp":"2024-09-18T07:18:26.438533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:49:22.011040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048630800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"26'7543","reported_seq":7572,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:26.660315+0000","last_change":"2024-09-18T07:18:26.660266+0000","last_active":"2024-09-18T07:18:26.660315+0000","last_peered":"2024-09-18T07:18:26.660315+0000","last_clean":"2024-09-18T07:18:26.660315+0000","last_became_active":"2024-09-18T07:14:22.475086+0000","last_became_peered":"2024-09-18T07:14:22.475086+0000","last_unstale":"2024-09-18T07:18:26.660315+0000","last_undegraded":"2024-09-18T07:18:26.660315+0000","last_fullsized":"2024-09-18T07:18:26.660315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'7543","last_scrub_stamp":"2024-09-18T07:18:26.660201+0000","last_deep_scrub":"26'7543","last_deep_scrub_stamp":"2024-09-18T07:18:26.660201+0000","last_clean_scrub_stamp":"2024-09-18T07:18:26.660201+0000","objects_scrubbed":31,"log_size":7543,"log_dups_size":0,"ondisk_log_size":7543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:12:55.254349+0000","scrub_duration":224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7506,"num_write_kb":189960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:23.367033+0000","last_change":"2024-09-18T07:18:23.366985+0000","last_active":"2024-09-18T07:18:23.367033+0000","last_peered":"2024-09-18T07:18:23.367033+0000","last_clean":"2024-09-18T07:18:23.367033+0000","last_became_active":"2024-09-18T07:14:13.956929+0000","last_became_peered":"2024-09-18T07:14:13.956929+0000","last_unstale":"2024-09-18T07:18:23.367033+0000","last_undegraded":"2024-09-18T07:18:23.367033+0000","last_fullsized":"2024-09-18T07:18:23.367033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:18:23.366924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:18:23.366924+0000","last_clean_scrub_stamp":"2024-09-18T07:18:23.366924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:09.256470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045339399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"26'5299","reported_seq":5327,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:25.045693+0000","last_change":"2024-09-18T07:18:25.045643+0000","last_active":"2024-09-18T07:18:25.045693+0000","last_peered":"2024-09-18T07:18:25.045693+0000","last_clean":"2024-09-18T07:18:25.045693+0000","last_became_active":"2024-09-18T07:14:22.288096+0000","last_became_peered":"2024-09-18T07:14:22.288096+0000","last_unstale":"2024-09-18T07:18:25.045693+0000","last_undegraded":"2024-09-18T07:18:25.045693+0000","last_fullsized":"2024-09-18T07:18:25.045693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5299","last_scrub_stamp":"2024-09-18T07:18:25.045558+0000","last_deep_scrub":"26'5299","last_deep_scrub_stamp":"2024-09-18T07:18:25.045558+0000","last_clean_scrub_stamp":"2024-09-18T07:18:25.045558+0000","objects_scrubbed":29,"log_size":5299,"log_dups_size":0,"ondisk_log_size":5299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:20:41.300968+0000","scrub_duration":253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5267,"num_write_kb":201861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"26'3631","reported_seq":3669,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:25.067159+0000","last_change":"2024-09-18T07:18:25.067114+0000","last_active":"2024-09-18T07:18:25.067159+0000","last_peered":"2024-09-18T07:18:25.067159+0000","last_clean":"2024-09-18T07:18:25.067159+0000","last_became_active":"2024-09-18T07:14:22.283205+0000","last_became_peered":"2024-09-18T07:14:22.283205+0000","last_unstale":"2024-09-18T07:18:25.067159+0000","last_undegraded":"2024-09-18T07:18:25.067159+0000","last_fullsized":"2024-09-18T07:18:25.067159+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3631","last_scrub_stamp":"2024-09-18T07:18:25.067047+0000","last_deep_scrub":"26'3631","last_deep_scrub_stamp":"2024-09-18T07:18:25.067047+0000","last_clean_scrub_stamp":"2024-09-18T07:18:25.067047+0000","objects_scrubbed":23,"log_size":3631,"log_dups_size":0,"ondisk_log_size":3631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:34:30.475861+0000","scrub_duration":198,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92283408,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":37,"num_read_kb":29,"num_write":3631,"num_write_kb":171797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"26'3","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:26.771890+0000","last_change":"2024-09-18T07:18:26.771890+0000","last_active":"2024-09-18T07:18:26.771890+0000","last_peered":"2024-09-18T07:18:26.771890+0000","last_clean":"2024-09-18T07:18:26.771890+0000","last_became_active":"2024-09-18T07:14:13.243579+0000","last_became_peered":"2024-09-18T07:14:13.243579+0000","last_unstale":"2024-09-18T07:18:26.771890+0000","last_undegraded":"2024-09-18T07:18:26.771890+0000","last_fullsized":"2024-09-18T07:18:26.771890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3","last_scrub_stamp":"2024-09-18T07:18:26.771828+0000","last_deep_scrub":"26'3","last_deep_scrub_stamp":"2024-09-18T07:18:26.771828+0000","last_clean_scrub_stamp":"2024-09-18T07:18:26.771828+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:00:55.339907+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00096575000000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"26'4988","reported_seq":5025,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:25.652911+0000","last_change":"2024-09-18T07:18:25.652911+0000","last_active":"2024-09-18T07:18:25.652911+0000","last_peered":"2024-09-18T07:18:25.652911+0000","last_clean":"2024-09-18T07:18:25.652911+0000","last_became_active":"2024-09-18T07:14:22.474980+0000","last_became_peered":"2024-09-18T07:14:22.474980+0000","last_unstale":"2024-09-18T07:18:25.652911+0000","last_undegraded":"2024-09-18T07:18:25.652911+0000","last_fullsized":"2024-09-18T07:18:25.652911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4988","last_scrub_stamp":"2024-09-18T07:18:25.652825+0000","last_deep_scrub":"26'4988","last_deep_scrub_stamp":"2024-09-18T07:18:25.652825+0000","last_clean_scrub_stamp":"2024-09-18T07:18:25.652825+0000","objects_scrubbed":30,"log_size":4988,"log_dups_size":0,"ondisk_log_size":4988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:28:52.606784+0000","scrub_duration":191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124452864,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4927,"num_write_kb":170572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:23.737655+0000","last_change":"2024-09-18T07:18:23.737608+0000","last_active":"2024-09-18T07:18:23.737655+0000","last_peered":"2024-09-18T07:18:23.737655+0000","last_clean":"2024-09-18T07:18:23.737655+0000","last_became_active":"2024-09-18T07:14:13.392213+0000","last_became_peered":"2024-09-18T07:14:13.392213+0000","last_unstale":"2024-09-18T07:18:23.737655+0000","last_undegraded":"2024-09-18T07:18:23.737655+0000","last_fullsized":"2024-09-18T07:18:23.737655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:18:23.737546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:18:23.737546+0000","last_clean_scrub_stamp":"2024-09-18T07:18:23.737546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:36:19.791872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000312059,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"26'3082","reported_seq":3121,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:23.526665+0000","last_change":"2024-09-18T07:18:23.526614+0000","last_active":"2024-09-18T07:18:23.526665+0000","last_peered":"2024-09-18T07:18:23.526665+0000","last_clean":"2024-09-18T07:18:23.526665+0000","last_became_active":"2024-09-18T07:14:22.286007+0000","last_became_peered":"2024-09-18T07:14:22.286007+0000","last_unstale":"2024-09-18T07:18:23.526665+0000","last_undegraded":"2024-09-18T07:18:23.526665+0000","last_fullsized":"2024-09-18T07:18:23.526665+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3082","last_scrub_stamp":"2024-09-18T07:18:23.526530+0000","last_deep_scrub":"26'3082","last_deep_scrub_stamp":"2024-09-18T07:18:23.526530+0000","last_clean_scrub_stamp":"2024-09-18T07:18:23.526530+0000","objects_scrubbed":24,"log_size":3082,"log_dups_size":0,"ondisk_log_size":3082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:59:59.245914+0000","scrub_duration":169,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92405760,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":3056,"num_write_kb":144258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:31.343954+0000","last_change":"2024-09-18T07:18:31.343954+0000","last_active":"2024-09-18T07:18:31.343954+0000","last_peered":"2024-09-18T07:18:31.343954+0000","last_clean":"2024-09-18T07:18:31.343954+0000","last_became_active":"2024-09-18T07:14:13.251213+0000","last_became_peered":"2024-09-18T07:14:13.251213+0000","last_unstale":"2024-09-18T07:18:31.343954+0000","last_undegraded":"2024-09-18T07:18:31.343954+0000","last_fullsized":"2024-09-18T07:18:31.343954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T07:18:31.343880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T07:18:31.343880+0000","last_clean_scrub_stamp":"2024-09-18T07:18:31.343880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:27:46.644102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046971900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"26'5359","reported_seq":5392,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:24.401960+0000","last_change":"2024-09-18T07:18:24.401882+0000","last_active":"2024-09-18T07:18:24.401960+0000","last_peered":"2024-09-18T07:18:24.401960+0000","last_clean":"2024-09-18T07:18:24.401960+0000","last_became_active":"2024-09-18T07:14:22.283928+0000","last_became_peered":"2024-09-18T07:14:22.283928+0000","last_unstale":"2024-09-18T07:18:24.401960+0000","last_undegraded":"2024-09-18T07:18:24.401960+0000","last_fullsized":"2024-09-18T07:18:24.401960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5359","last_scrub_stamp":"2024-09-18T07:18:24.401752+0000","last_deep_scrub":"26'5359","last_deep_scrub_stamp":"2024-09-18T07:18:24.401752+0000","last_clean_scrub_stamp":"2024-09-18T07:18:24.401752+0000","objects_scrubbed":23,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:38:19.981269+0000","scrub_duration":173,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080896,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":18,"num_read_kb":13,"num_write":5351,"num_write_kb":144518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"26'2","reported_seq":42,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:27.775660+0000","last_change":"2024-09-18T07:18:27.775660+0000","last_active":"2024-09-18T07:18:27.775660+0000","last_peered":"2024-09-18T07:18:27.775660+0000","last_clean":"2024-09-18T07:18:27.775660+0000","last_became_active":"2024-09-18T07:14:13.243793+0000","last_became_peered":"2024-09-18T07:14:13.243793+0000","last_unstale":"2024-09-18T07:18:27.775660+0000","last_undegraded":"2024-09-18T07:18:27.775660+0000","last_fullsized":"2024-09-18T07:18:27.775660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'2","last_scrub_stamp":"2024-09-18T07:18:27.775583+0000","last_deep_scrub":"26'2","last_deep_scrub_stamp":"2024-09-18T07:18:27.775583+0000","last_clean_scrub_stamp":"2024-09-18T07:18:27.775583+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:35:11.996754+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00039863000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"26'74","reported_seq":133,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:28.412533+0000","last_change":"2024-09-18T07:18:28.412533+0000","last_active":"2024-09-18T07:18:28.412533+0000","last_peered":"2024-09-18T07:18:28.412533+0000","last_clean":"2024-09-18T07:18:28.412533+0000","last_became_active":"2024-09-18T07:14:06.880771+0000","last_became_peered":"2024-09-18T07:14:06.880771+0000","last_unstale":"2024-09-18T07:18:28.412533+0000","last_undegraded":"2024-09-18T07:18:28.412533+0000","last_fullsized":"2024-09-18T07:18:28.412533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'74","last_scrub_stamp":"2024-09-18T07:18:28.412457+0000","last_deep_scrub":"26'74","last_deep_scrub_stamp":"2024-09-18T07:18:28.412457+0000","last_clean_scrub_stamp":"2024-09-18T07:18:28.412457+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:17:21.426047+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"26'5269","reported_seq":5298,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:26.438618+0000","last_change":"2024-09-18T07:18:26.438570+0000","last_active":"2024-09-18T07:18:26.438618+0000","last_peered":"2024-09-18T07:18:26.438618+0000","last_clean":"2024-09-18T07:18:26.438618+0000","last_became_active":"2024-09-18T07:14:22.279712+0000","last_became_peered":"2024-09-18T07:14:22.279712+0000","last_unstale":"2024-09-18T07:18:26.438618+0000","last_undegraded":"2024-09-18T07:18:26.438618+0000","last_fullsized":"2024-09-18T07:18:26.438618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5269","last_scrub_stamp":"2024-09-18T07:18:26.438482+0000","last_deep_scrub":"26'5269","last_deep_scrub_stamp":"2024-09-18T07:18:26.438482+0000","last_clean_scrub_stamp":"2024-09-18T07:18:26.438482+0000","objects_scrubbed":33,"log_size":5269,"log_dups_size":0,"ondisk_log_size":5269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:15:08.835831+0000","scrub_duration":187,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136970240,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5162,"num_write_kb":248325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"26'8297","reported_seq":8454,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:27.102659+0000","last_change":"2024-09-18T07:18:27.102597+0000","last_active":"2024-09-18T07:18:27.102659+0000","last_peered":"2024-09-18T07:18:27.102659+0000","last_clean":"2024-09-18T07:18:27.102659+0000","last_became_active":"2024-09-18T07:14:22.475044+0000","last_became_peered":"2024-09-18T07:14:22.475044+0000","last_unstale":"2024-09-18T07:18:27.102659+0000","last_undegraded":"2024-09-18T07:18:27.102659+0000","last_fullsized":"2024-09-18T07:18:27.102659+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'8297","last_scrub_stamp":"2024-09-18T07:18:27.102474+0000","last_deep_scrub":"26'8297","last_deep_scrub_stamp":"2024-09-18T07:18:27.102474+0000","last_clean_scrub_stamp":"2024-09-18T07:18:27.102474+0000","objects_scrubbed":24,"log_size":8297,"log_dups_size":0,"ondisk_log_size":8297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:44:49.235220+0000","scrub_duration":193,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":409,"num_read_kb":432,"num_write":5209,"num_write_kb":145235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"26'6974","reported_seq":7005,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:26.405601+0000","last_change":"2024-09-18T07:18:26.405519+0000","last_active":"2024-09-18T07:18:26.405601+0000","last_peered":"2024-09-18T07:18:26.405601+0000","last_clean":"2024-09-18T07:18:26.405601+0000","last_became_active":"2024-09-18T07:14:22.285050+0000","last_became_peered":"2024-09-18T07:14:22.285050+0000","last_unstale":"2024-09-18T07:18:26.405601+0000","last_undegraded":"2024-09-18T07:18:26.405601+0000","last_fullsized":"2024-09-18T07:18:26.405601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'6974","last_scrub_stamp":"2024-09-18T07:18:26.405413+0000","last_deep_scrub":"26'6974","last_deep_scrub_stamp":"2024-09-18T07:18:26.405413+0000","last_clean_scrub_stamp":"2024-09-18T07:18:26.405413+0000","objects_scrubbed":35,"log_size":6974,"log_dups_size":0,"ondisk_log_size":6974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:03:22.360536+0000","scrub_duration":369,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146120704,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":6915,"num_write_kb":222193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"26'4839","reported_seq":4882,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:27.446032+0000","last_change":"2024-09-18T07:18:27.446032+0000","last_active":"2024-09-18T07:18:27.446032+0000","last_peered":"2024-09-18T07:18:27.446032+0000","last_clean":"2024-09-18T07:18:27.446032+0000","last_became_active":"2024-09-18T07:14:22.284397+0000","last_became_peered":"2024-09-18T07:14:22.284397+0000","last_unstale":"2024-09-18T07:18:27.446032+0000","last_undegraded":"2024-09-18T07:18:27.446032+0000","last_fullsized":"2024-09-18T07:18:27.446032+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4839","last_scrub_stamp":"2024-09-18T07:18:27.445952+0000","last_deep_scrub":"26'4839","last_deep_scrub_stamp":"2024-09-18T07:18:27.445952+0000","last_clean_scrub_stamp":"2024-09-18T07:18:27.445952+0000","objects_scrubbed":26,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:14:48.264479+0000","scrub_duration":171,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104988672,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":64,"num_read_kb":50,"num_write":4839,"num_write_kb":163526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"26'4951","reported_seq":4989,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:26.036936+0000","last_change":"2024-09-18T07:18:26.036936+0000","last_active":"2024-09-18T07:18:26.036936+0000","last_peered":"2024-09-18T07:18:26.036936+0000","last_clean":"2024-09-18T07:18:26.036936+0000","last_became_active":"2024-09-18T07:14:22.287513+0000","last_became_peered":"2024-09-18T07:14:22.287513+0000","last_unstale":"2024-09-18T07:18:26.036936+0000","last_undegraded":"2024-09-18T07:18:26.036936+0000","last_fullsized":"2024-09-18T07:18:26.036936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'4951","last_scrub_stamp":"2024-09-18T07:18:26.036811+0000","last_deep_scrub":"26'4951","last_deep_scrub_stamp":"2024-09-18T07:18:26.036811+0000","last_clean_scrub_stamp":"2024-09-18T07:18:26.036811+0000","objects_scrubbed":25,"log_size":4951,"log_dups_size":0,"ondisk_log_size":4951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:42:01.661968+0000","scrub_duration":233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794368,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":43,"num_read_kb":96,"num_write":4998,"num_write_kb":181316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"26'3071","reported_seq":3100,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:29.616875+0000","last_change":"2024-09-18T07:18:29.616875+0000","last_active":"2024-09-18T07:18:29.616875+0000","last_peered":"2024-09-18T07:18:29.616875+0000","last_clean":"2024-09-18T07:18:29.616875+0000","last_became_active":"2024-09-18T07:14:22.288690+0000","last_became_peered":"2024-09-18T07:14:22.288690+0000","last_unstale":"2024-09-18T07:18:29.616875+0000","last_undegraded":"2024-09-18T07:18:29.616875+0000","last_fullsized":"2024-09-18T07:18:29.616875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3071","last_scrub_stamp":"2024-09-18T07:18:29.616808+0000","last_deep_scrub":"26'3071","last_deep_scrub_stamp":"2024-09-18T07:18:29.616808+0000","last_clean_scrub_stamp":"2024-09-18T07:18:29.616808+0000","objects_scrubbed":23,"log_size":3071,"log_dups_size":0,"ondisk_log_size":3071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:27:04.401397+0000","scrub_duration":220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92275200,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":3030,"num_write_kb":128003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"26'3922","reported_seq":4041,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:27.647333+0000","last_change":"2024-09-18T07:18:27.647333+0000","last_active":"2024-09-18T07:18:27.647333+0000","last_peered":"2024-09-18T07:18:27.647333+0000","last_clean":"2024-09-18T07:18:27.647333+0000","last_became_active":"2024-09-18T07:14:22.288767+0000","last_became_peered":"2024-09-18T07:14:22.288767+0000","last_unstale":"2024-09-18T07:18:27.647333+0000","last_undegraded":"2024-09-18T07:18:27.647333+0000","last_fullsized":"2024-09-18T07:18:27.647333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3922","last_scrub_stamp":"2024-09-18T07:18:27.647234+0000","last_deep_scrub":"26'3922","last_deep_scrub_stamp":"2024-09-18T07:18:27.647234+0000","last_clean_scrub_stamp":"2024-09-18T07:18:27.647234+0000","objects_scrubbed":26,"log_size":3922,"log_dups_size":0,"ondisk_log_size":3922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:16:00.625724+0000","scrub_duration":215,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106366632,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":300,"num_read_kb":235,"num_write":4144,"num_write_kb":154845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"26'5151","reported_seq":5179,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:34.612266+0000","last_change":"2024-09-18T07:18:34.612266+0000","last_active":"2024-09-18T07:18:34.612266+0000","last_peered":"2024-09-18T07:18:34.612266+0000","last_clean":"2024-09-18T07:18:34.612266+0000","last_became_active":"2024-09-18T07:14:22.289527+0000","last_became_peered":"2024-09-18T07:14:22.289527+0000","last_unstale":"2024-09-18T07:18:34.612266+0000","last_undegraded":"2024-09-18T07:18:34.612266+0000","last_fullsized":"2024-09-18T07:18:34.612266+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'5151","last_scrub_stamp":"2024-09-18T07:18:34.612206+0000","last_deep_scrub":"26'5151","last_deep_scrub_stamp":"2024-09-18T07:18:34.612206+0000","last_clean_scrub_stamp":"2024-09-18T07:18:34.612206+0000","objects_scrubbed":32,"log_size":5151,"log_dups_size":0,"ondisk_log_size":5151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:38:34.240329+0000","scrub_duration":284,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130834432,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5068,"num_write_kb":190489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"26'3047","reported_seq":3379,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:28.150093+0000","last_change":"2024-09-18T07:18:28.150093+0000","last_active":"2024-09-18T07:18:28.150093+0000","last_peered":"2024-09-18T07:18:28.150093+0000","last_clean":"2024-09-18T07:18:28.150093+0000","last_became_active":"2024-09-18T07:14:22.284989+0000","last_became_peered":"2024-09-18T07:14:22.284989+0000","last_unstale":"2024-09-18T07:18:28.150093+0000","last_undegraded":"2024-09-18T07:18:28.150093+0000","last_fullsized":"2024-09-18T07:18:28.150093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3047","last_scrub_stamp":"2024-09-18T07:18:28.150031+0000","last_deep_scrub":"26'3047","last_deep_scrub_stamp":"2024-09-18T07:18:28.150031+0000","last_clean_scrub_stamp":"2024-09-18T07:18:28.150031+0000","objects_scrubbed":18,"log_size":3047,"log_dups_size":0,"ondisk_log_size":3047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:39:52.605845+0000","scrub_duration":143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71434240,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1022,"num_read_kb":916,"num_write":3192,"num_write_kb":144514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"26'3087","reported_seq":3137,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:30.568558+0000","last_change":"2024-09-18T07:18:30.568558+0000","last_active":"2024-09-18T07:18:30.568558+0000","last_peered":"2024-09-18T07:18:30.568558+0000","last_clean":"2024-09-18T07:18:30.568558+0000","last_became_active":"2024-09-18T07:14:22.290299+0000","last_became_peered":"2024-09-18T07:14:22.290299+0000","last_unstale":"2024-09-18T07:18:30.568558+0000","last_undegraded":"2024-09-18T07:18:30.568558+0000","last_fullsized":"2024-09-18T07:18:30.568558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3087","last_scrub_stamp":"2024-09-18T07:18:30.568477+0000","last_deep_scrub":"26'3087","last_deep_scrub_stamp":"2024-09-18T07:18:30.568477+0000","last_clean_scrub_stamp":"2024-09-18T07:18:30.568477+0000","objects_scrubbed":22,"log_size":3087,"log_dups_size":0,"ondisk_log_size":3087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:41:34.433772+0000","scrub_duration":192,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":3095,"num_write_kb":160450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"26'3736","reported_seq":3763,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:25.592812+0000","last_change":"2024-09-18T07:18:25.592812+0000","last_active":"2024-09-18T07:18:25.592812+0000","last_peered":"2024-09-18T07:18:25.592812+0000","last_clean":"2024-09-18T07:18:25.592812+0000","last_became_active":"2024-09-18T07:14:22.290308+0000","last_became_peered":"2024-09-18T07:14:22.290308+0000","last_unstale":"2024-09-18T07:18:25.592812+0000","last_undegraded":"2024-09-18T07:18:25.592812+0000","last_fullsized":"2024-09-18T07:18:25.592812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'3736","last_scrub_stamp":"2024-09-18T07:18:25.592689+0000","last_deep_scrub":"26'3736","last_deep_scrub_stamp":"2024-09-18T07:18:25.592689+0000","last_clean_scrub_stamp":"2024-09-18T07:18:25.592689+0000","objects_scrubbed":29,"log_size":3736,"log_dups_size":0,"ondisk_log_size":3736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:07:51.363876+0000","scrub_duration":252,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120979456,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3652,"num_write_kb":176387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"26'7773","reported_seq":7811,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T07:18:33.622950+0000","last_change":"2024-09-18T07:18:33.622950+0000","last_active":"2024-09-18T07:18:33.622950+0000","last_peered":"2024-09-18T07:18:33.622950+0000","last_clean":"2024-09-18T07:18:33.622950+0000","last_became_active":"2024-09-18T07:14:22.287927+0000","last_became_peered":"2024-09-18T07:14:22.287927+0000","last_unstale":"2024-09-18T07:18:33.622950+0000","last_undegraded":"2024-09-18T07:18:33.622950+0000","last_fullsized":"2024-09-18T07:18:33.622950+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'7773","last_scrub_stamp":"2024-09-18T07:18:33.622869+0000","last_deep_scrub":"26'7773","last_deep_scrub_stamp":"2024-09-18T07:18:33.622869+0000","last_clean_scrub_stamp":"2024-09-18T07:18:33.622869+0000","objects_scrubbed":28,"log_size":7773,"log_dups_size":0,"ondisk_log_size":7773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:11:04.771477+0000","scrub_duration":267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":31,"num_read_kb":87,"num_write":7741,"num_write_kb":165450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3581698296,"num_objects":881,"num_object_clones":0,"num_object_copies":1762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":0,"num_read":2575,"num_read_kb":2591,"num_write":165342,"num_write_kb":5659041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6865731584,"data_stored":6861464696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":169085,"ondisk_log_size":169085,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1313876,"kb_used_data":1043972,"kb_used_omap":1,"kb_used_meta":269886,"kb_avail":93057964,"statfs":{"total":96636764160,"available":95291355136,"internally_reserved":0,"allocated":1069027328,"data_stored":1068302773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":276363722},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408844,"kb_used_data":311036,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93962996,"statfs":{"total":96636764160,"available":96218107904,"internally_reserved":0,"allocated":318500864,"data_stored":318134177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":100137414},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542202,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1678256,"kb_used_data":1300128,"kb_used_omap":1,"kb_used_meta":378110,"kb_avail":92693584,"statfs":{"total":96636764160,"available":94918230016,"internally_reserved":0,"allocated":1331331072,"data_stored":1330392625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":387185093},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542202,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1384284,"kb_used_data":1082444,"kb_used_omap":1,"kb_used_meta":301822,"kb_avail":92987556,"statfs":{"total":96636764160,"available":95219257344,"internally_reserved":0,"allocated":1108422656,"data_stored":1107555517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":309066183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542202,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436752,"kb_used_data":338944,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":93935088,"statfs":{"total":96636764160,"available":96189530112,"internally_reserved":0,"allocated":347078656,"data_stored":346704641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":100137413},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542202,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902312,"kb_used_data":676824,"kb_used_omap":1,"kb_used_meta":225470,"kb_avail":93469528,"statfs":{"total":96636764160,"available":95712796672,"internally_reserved":0,"allocated":693067776,"data_stored":692416132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":230881733},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542202,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1276100,"kb_used_data":1002036,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":93095740,"statfs":{"total":96636764160,"available":95330037760,"internally_reserved":0,"allocated":1026084864,"data_stored":1025252949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":280623557},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1182048,"kb_used_data":952400,"kb_used_omap":1,"kb_used_meta":229630,"kb_avail":93189792,"statfs":{"total":96636764160,"available":95426347008,"internally_reserved":0,"allocated":975257600,"data_stored":974458280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235141573},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1025306624,"data_stored":1024637208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":692817920,"data_stored":692329300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346832896,"data_stored":346617828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1108176896,"data_stored":1107468704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1331085312,"data_stored":1330305812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":975007744,"data_stored":974371448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":318255104,"data_stored":318047364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1068249088,"data_stored":1067687032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T07:18:46.331 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-18T07:18:46.429 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.426+0000 7f2309a71640 1 Processor -- start 2024-09-18T07:18:46.429 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.426+0000 7f2309a71640 1 -- start start 2024-09-18T07:18:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.426+0000 7f2309a71640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23040aae10 con 0x7f2304072aa0 2024-09-18T07:18:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.426+0000 7f2309a71640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23040739b0 con 0x7f230413d610 2024-09-18T07:18:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2309a71640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2304073f60 con 0x7f2304070c30 2024-09-18T07:18:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f22fbfff640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f2304070c30 0x7f230413cf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:54088/0 (socket says 172.21.15.70:54088) 2024-09-18T07:18:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f23037fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2304072aa0 0x7f2304140570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42224/0 (socket says 172.21.15.70:42224) 2024-09-18T07:18:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f22fbfff640 1 -- 172.21.15.70:0/1007271552 learned_addr learned my addr 172.21.15.70:0/1007271552 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2302ffd640 1 -- 172.21.15.70:0/1007271552 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 139954483 0 0) 0x7f23040aae10 con 0x7f2304072aa0 2024-09-18T07:18:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2302ffd640 1 -- 172.21.15.70:0/1007271552 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22f0003520 con 0x7f2304072aa0 2024-09-18T07:18:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2302ffd640 1 -- 172.21.15.70:0/1007271552 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2916527106 0 0) 0x7f2304073f60 con 0x7f2304070c30 2024-09-18T07:18:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2302ffd640 1 -- 172.21.15.70:0/1007271552 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23040aae10 con 0x7f2304070c30 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2302ffd640 1 -- 172.21.15.70:0/1007271552 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3311980570 0 0) 0x7f23040739b0 con 0x7f230413d610 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2302ffd640 1 -- 172.21.15.70:0/1007271552 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2304073f60 con 0x7f230413d610 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2302ffd640 1 -- 172.21.15.70:0/1007271552 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3231633717 0 0) 0x7f22f0003520 con 0x7f2304072aa0 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2302ffd640 1 -- 172.21.15.70:0/1007271552 >> v1:172.21.15.70:6790/0 conn(0x7f2304070c30 legacy=0x7f230413cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2302ffd640 1 -- 172.21.15.70:0/1007271552 >> v1:172.21.15.172:6789/0 conn(0x7f230413d610 legacy=0x7f230414b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2302ffd640 1 -- 172.21.15.70:0/1007271552 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2304140c80 con 0x7f2304072aa0 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2302ffd640 1 -- 172.21.15.70:0/1007271552 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f22f4001690 con 0x7f2304072aa0 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2302ffd640 1 -- 172.21.15.70:0/1007271552 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f22f40018b0 con 0x7f2304072aa0 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.427+0000 7f2302ffd640 1 -- 172.21.15.70:0/1007271552 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f22f4001d00 con 0x7f2304072aa0 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.428+0000 7f2309a71640 1 -- 172.21.15.70:0/1007271552 >> v1:172.21.15.70:6789/0 conn(0x7f2304072aa0 legacy=0x7f2304140570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.428+0000 7f2309a71640 1 -- 172.21.15.70:0/1007271552 shutdown_connections 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.428+0000 7f2309a71640 1 -- 172.21.15.70:0/1007271552 >> 172.21.15.70:0/1007271552 conn(0x7f2304076ca0 msgr2=0x7f23040b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.428+0000 7f2309a71640 1 -- 172.21.15.70:0/1007271552 shutdown_connections 2024-09-18T07:18:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.428+0000 7f2309a71640 1 -- 172.21.15.70:0/1007271552 wait complete. 2024-09-18T07:18:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.429+0000 7f2309a71640 1 Processor -- start 2024-09-18T07:18:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.429+0000 7f2309a71640 1 -- start start 2024-09-18T07:18:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.429+0000 7f2309a71640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23040af380 con 0x7f2304072aa0 2024-09-18T07:18:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.429+0000 7f2309a71640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2304139590 con 0x7f2304070c30 2024-09-18T07:18:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.429+0000 7f2309a71640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23040ad090 con 0x7f230413d610 2024-09-18T07:18:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f22fbfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2304072aa0 0x7f2304138770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42226/0 (socket says 172.21.15.70:42226) 2024-09-18T07:18:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f22fbfff640 1 -- 172.21.15.70:0/2261771088 learned_addr learned my addr 172.21.15.70:0/2261771088 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-18T07:18:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4164831698 0 0) 0x7f23040af380 con 0x7f2304072aa0 2024-09-18T07:18:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22e4003550 con 0x7f2304072aa0 2024-09-18T07:18:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1376550091 0 0) 0x7f23040ad090 con 0x7f230413d610 2024-09-18T07:18:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23040af380 con 0x7f230413d610 2024-09-18T07:18:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3303962848 0 0) 0x7f2304139590 con 0x7f2304070c30 2024-09-18T07:18:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23040ad090 con 0x7f2304070c30 2024-09-18T07:18:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3197726147 0 0) 0x7f22e4003550 con 0x7f2304072aa0 2024-09-18T07:18:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2304139590 con 0x7f2304072aa0 2024-09-18T07:18:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f22ec001970 con 0x7f2304072aa0 2024-09-18T07:18:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3512728324 0 0) 0x7f23040af380 con 0x7f230413d610 2024-09-18T07:18:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f22e4003550 con 0x7f230413d610 2024-09-18T07:18:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f22fc001b50 con 0x7f230413d610 2024-09-18T07:18:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2470452956 0 0) 0x7f2304139590 con 0x7f2304072aa0 2024-09-18T07:18:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 >> v1:172.21.15.70:6790/0 conn(0x7f230413d610 legacy=0x7f2304138e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 >> v1:172.21.15.172:6789/0 conn(0x7f2304070c30 legacy=0x7f23040ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230414f9f0 con 0x7f2304072aa0 2024-09-18T07:18:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2309a71640 1 -- 172.21.15.70:0/2261771088 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f23040adf20 con 0x7f2304072aa0 2024-09-18T07:18:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2309a71640 1 -- 172.21.15.70:0/2261771088 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f23040ad550 con 0x7f2304072aa0 2024-09-18T07:18:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f22ec001c70 con 0x7f2304072aa0 2024-09-18T07:18:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.430+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 945209547 0 0) 0x7f22ec004c30 con 0x7f2304072aa0 2024-09-18T07:18:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.431+0000 7f2309a71640 1 -- 172.21.15.70:0/2261771088 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e8005080 con 0x7f2304072aa0 2024-09-18T07:18:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.432+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) ==== 101406+0+0 (unknown 1884164251 0 0) 0x7f22ec01d860 con 0x7f2304072aa0 2024-09-18T07:18:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.432+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 4525+0+0 (unknown 3399314850 0 0) 0x7f22ec0960d0 con 0x7f2304072aa0 2024-09-18T07:18:46.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.436+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (unknown 1092875540 0 2276561608) 0x7f22ec05e8d0 con 0x7f2304072aa0 2024-09-18T07:18:46.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.611+0000 7f2309a71640 1 -- 172.21.15.70:0/2261771088 --> v1:172.21.15.70:6789/0 -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f22e8005370 con 0x7f2304072aa0 2024-09-18T07:18:46.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.614+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v7)=0 v7) ==== 123+0+0 (unknown 3414686427 0 0) 0x7f22ec05e1d0 con 0x7f2304072aa0 2024-09-18T07:18:46.618 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.615+0000 7f2300ff9640 1 -- 172.21.15.70:0/2261771088 <== mon.0 v1:172.21.15.70:6789/0 11 ==== config(1 keys) ==== 35+0+0 (unknown 2555425225 0 0) 0x7f22ec05dce0 con 0x7f2304072aa0 2024-09-18T07:18:46.620 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.617+0000 7f2309a71640 1 -- 172.21.15.70:0/2261771088 >> v1:172.21.15.70:6816/1657990791 conn(0x7f22e407b8b0 legacy=0x7f22e407dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.620 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.617+0000 7f2309a71640 1 -- 172.21.15.70:0/2261771088 >> v1:172.21.15.70:6789/0 conn(0x7f2304072aa0 legacy=0x7f2304138770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-18T07:18:46.620 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.617+0000 7f2309a71640 1 -- 172.21.15.70:0/2261771088 shutdown_connections 2024-09-18T07:18:46.620 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.617+0000 7f2309a71640 1 -- 172.21.15.70:0/2261771088 >> 172.21.15.70:0/2261771088 conn(0x7f2304076ca0 msgr2=0x7f230413ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-18T07:18:46.620 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.617+0000 7f2309a71640 1 -- 172.21.15.70:0/2261771088 shutdown_connections 2024-09-18T07:18:46.620 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-18T07:18:46.617+0000 7f2309a71640 1 -- 172.21.15.70:0/2261771088 wait complete. 2024-09-18T07:18:46.630 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-18T07:18:46.630 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-18T07:18:46.631 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-18T07:18:46.631 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T07:18:46.685 INFO:tasks.ceph.osd.0:Stopped 2024-09-18T07:18:46.686 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-18T07:18:46.686 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T07:18:46.736 INFO:tasks.ceph.osd.1:Stopped 2024-09-18T07:18:46.736 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-18T07:18:46.736 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T07:18:46.773 INFO:tasks.ceph.osd.2:Stopped 2024-09-18T07:18:46.774 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-18T07:18:46.774 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T07:18:46.853 INFO:tasks.ceph.osd.3:Stopped 2024-09-18T07:18:46.853 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-18T07:18:46.853 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T07:18:46.904 INFO:tasks.ceph.osd.4:Stopped 2024-09-18T07:18:46.904 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-18T07:18:46.904 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T07:18:46.964 INFO:tasks.ceph.osd.5:Stopped 2024-09-18T07:18:46.964 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-18T07:18:46.964 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T07:18:47.014 INFO:tasks.ceph.osd.6:Stopped 2024-09-18T07:18:47.014 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-18T07:18:47.015 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T07:18:47.062 INFO:tasks.ceph.osd.7:Stopped 2024-09-18T07:18:47.062 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-18T07:18:47.062 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-18T07:18:47.063 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T07:18:47.091 INFO:tasks.ceph.mgr.y:Stopped 2024-09-18T07:18:47.091 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-18T07:18:47.091 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T07:18:47.119 INFO:tasks.ceph.mgr.x:Stopped 2024-09-18T07:18:47.119 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-18T07:18:47.119 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-18T07:18:47.119 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T07:18:47.143 INFO:tasks.ceph.mon.a:Stopped 2024-09-18T07:18:47.144 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-18T07:18:47.144 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T07:18:47.194 INFO:tasks.ceph.mon.c:Stopped 2024-09-18T07:18:47.195 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-18T07:18:47.195 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T07:18:47.206 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-18T07:18:47.206 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-18T07:18:47.206 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-18T07:18:47.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-18T07:18:47.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-18T07:18:47.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-18T07:18:47.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-18T07:18:47.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-18T07:18:47.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-09-18T07:18:47.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-18T07:18:47.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-18T07:18:47.207 INFO:tasks.ceph.mon.b:Stopped 2024-09-18T07:18:47.207 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-18T07:18:47.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | head -n 1 2024-09-18T07:18:47.263 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-18T07:18:47.263 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-18T07:18:47.696 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-18T07:18:47.697 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-18T07:18:47.787 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-18T07:18:47.788 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-18T07:18:47.863 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-18T07:18:47.863 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-18T07:18:47.940 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi172.front.sepia.ceph.com 2024-09-18T07:18:47.940 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-18T07:18:48.136 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi172.front.sepia.ceph.com 2024-09-18T07:18:48.137 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-18T07:18:48.224 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi172.front.sepia.ceph.com 2024-09-18T07:18:48.224 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-18T07:18:48.282 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi172.front.sepia.ceph.com 2024-09-18T07:18:48.282 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-18T07:18:48.349 INFO:tasks.ceph:Archiving mon data... 2024-09-18T07:18:48.351 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi/7910278/data/mon.a.tgz 2024-09-18T07:18:48.351 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-18T07:18:48.369 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.Ppli89tWoR 2024-09-18T07:18:48.370 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Ppli89tWoR 2024-09-18T07:18:48.510 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.Ppli89tWoR 2024-09-18T07:18:48.596 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.Ppli89tWoR is 294KB 2024-09-18T07:18:48.691 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.Ppli89tWoR 2024-09-18T07:18:48.706 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi/7910278/data/mon.c.tgz 2024-09-18T07:18:48.707 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-18T07:18:48.761 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.DOWTJFz8km 2024-09-18T07:18:48.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.DOWTJFz8km 2024-09-18T07:18:48.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.DOWTJFz8km 2024-09-18T07:18:48.957 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.DOWTJFz8km is 295KB 2024-09-18T07:18:49.056 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.DOWTJFz8km 2024-09-18T07:18:49.072 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi/7910278/data/mon.b.tgz 2024-09-18T07:18:49.072 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2024-09-18T07:18:49.090 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.bjVaf5ipCH 2024-09-18T07:18:49.091 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.bjVaf5ipCH 2024-09-18T07:18:49.246 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.bjVaf5ipCH 2024-09-18T07:18:49.333 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.bjVaf5ipCH is 295KB 2024-09-18T07:18:49.406 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.bjVaf5ipCH 2024-09-18T07:18:49.421 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-18T07:18:49.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T07:18:49.424 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T07:18:49.502 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-18T07:18:49.541 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-18T07:18:49.541 INFO:tasks.ceph:Archiving crash dumps... 2024-09-18T07:18:49.543 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/crash to /home/teuthworker/archive/nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi/7910278/remote/smithi070/crash 2024-09-18T07:18:49.544 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T07:18:49.571 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/crash to /home/teuthworker/archive/nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi/7910278/remote/smithi172/crash 2024-09-18T07:18:49.572 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T07:18:49.605 INFO:tasks.ceph:Compressing logs... 2024-09-18T07:18:49.605 DEBUG:teuthology.orchestra.run.smithi070:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T07:18:49.613 DEBUG:teuthology.orchestra.run.smithi172:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T07:18:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-18T07:18:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39900.log 2024-09-18T07:18:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37746.log 2024-09-18T07:18:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37943.log 2024-09-18T07:18:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37746.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.39900.log.gz 2024-09-18T07:18:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31336.log 2024-09-18T07:18:49.663 INFO:teuthology.orchestra.run.smithi070.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.37746.log.gz 2024-09-18T07:18:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37943.log: /var/log/ceph/ceph-client.admin.31336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37745.log 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.37943.log.gz 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.1.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.31336.log.gz 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38067.log 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37065.log 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38067.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.37745.log.gz 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39650.log 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38067.log.gz 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37406.log 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37065.log.gz 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36893.log 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39650.log: /var/log/ceph/ceph-client.admin.37406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39272.log 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.39650.log.gz 2024-09-18T07:18:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.36893.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.37406.log.gz 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39524.log 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36893.log.gz 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36767.log 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39272.log: /var/log/ceph/ceph-client.admin.39524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.38762.log 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.39272.log.gz 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.39524.log.gz 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.36767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37401.log 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39146.log 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36767.log.gz 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.libcephsqlite.38762.log: /var/log/ceph/ceph-client.admin.37401.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.25501.log 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.37401.log.gz 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.38788.log 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.39146.log.gz 2024-09-18T07:18:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.38622.log 2024-09-18T07:18:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.tmp-client.admin.25501.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.25501.log.gz 2024-09-18T07:18:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.libcephsqlite.38788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39860.log 2024-09-18T07:18:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.libcephsqlite.38622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39398.log 2024-09-18T07:18:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37409.log 2024-09-18T07:18:49.666 INFO:teuthology.orchestra.run.smithi070.stderr: 86.3%/var/log/ceph/ceph-client.admin.39398.log: -- replaced with /var/log/ceph/ceph-client.admin.39860.log.gz 2024-09-18T07:18:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.38687.log 86.9% 2024-09-18T07:18:49.666 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39398.log.gz 2024-09-18T07:18:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36535.log 2024-09-18T07:18:49.666 INFO:teuthology.orchestra.run.smithi070.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.37409.log.gz 2024-09-18T07:18:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.libcephsqlite.38687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37074.log 2024-09-18T07:18:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.36535.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T07:18:49.667 INFO:teuthology.orchestra.run.smithi070.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.36535.log.gz 2024-09-18T07:18:49.667 INFO:teuthology.orchestra.run.smithi070.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.38622.log.gz 2024-09-18T07:18:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37404.log 2024-09-18T07:18:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37074.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36716.log 2024-09-18T07:18:49.667 INFO:teuthology.orchestra.run.smithi070.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.37074.log.gz 2024-09-18T07:18:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37075.log 2024-09-18T07:18:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.36716.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.37404.log.gz 2024-09-18T07:18:49.667 INFO:teuthology.orchestra.run.smithi070.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T07:18:49.667 INFO:teuthology.orchestra.run.smithi070.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.36716.log.gz 2024-09-18T07:18:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37408.log 2024-09-18T07:18:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39440.log 2024-09-18T07:18:49.667 INFO:teuthology.orchestra.run.smithi070.stderr: 86.1%/var/log/ceph/ceph-client.admin.37408.log: -- replaced with /var/log/ceph/ceph-client.admin.37075.log.gz 2024-09-18T07:18:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39062.log 2024-09-18T07:18:49.668 INFO:teuthology.orchestra.run.smithi070.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37408.log.gz 2024-09-18T07:18:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38978.log/var/log/ceph/ceph-client.admin.39440.log: 2024-09-18T07:18:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39062.log: 86.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.38150.log 2024-09-18T07:18:49.668 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39440.log.gz 2024-09-18T07:18:49.668 INFO:teuthology.orchestra.run.smithi070.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.39062.log.gz 2024-09-18T07:18:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38110.log 2024-09-18T07:18:49.669 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.libcephsqlite.38150.log: gzip 86.2% -5 --verbose -- /var/log/ceph/ceph-client.admin.36851.log 2024-09-18T07:18:49.669 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38978.log.gz 2024-09-18T07:18:49.669 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38110.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.38110.log.gz 2024-09-18T07:18:49.669 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-18T07:18:49.669 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37073.log 2024-09-18T07:18:49.669 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.36851.log: /var/log/ceph/ceph-mon.c.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36851.log.gz 2024-09-18T07:18:49.669 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39776.log 2024-09-18T07:18:49.669 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37073.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.37073.log.gz 2024-09-18T07:18:49.669 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39566.log 2024-09-18T07:18:49.669 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39776.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.39776.log.gz 2024-09-18T07:18:49.669 INFO:teuthology.orchestra.run.smithi070.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.38687.log.gz 2024-09-18T07:18:49.669 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37627.log 2024-09-18T07:18:49.669 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31638.log 2024-09-18T07:18:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37627.log: /var/log/ceph/ceph-client.admin.39566.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.39566.log.gz 2024-09-18T07:18:49.670 INFO:teuthology.orchestra.run.smithi070.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.37627.log.gz 2024-09-18T07:18:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39734.log 2024-09-18T07:18:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.31638.log: gzip -5 --verbose 87.1% -- replaced with /var/log/ceph/ceph-client.admin.31638.log.gz 2024-09-18T07:18:49.670 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.39020.log 2024-09-18T07:18:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39692.log 2024-09-18T07:18:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39020.log: /var/log/ceph/ceph-client.admin.39734.log: 86.5% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39734.log.gz 2024-09-18T07:18:49.670 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39020.log.gz 2024-09-18T07:18:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39608.log 2024-09-18T07:18:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39692.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-18T07:18:49.670 INFO:teuthology.orchestra.run.smithi070.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.39692.log.gz 2024-09-18T07:18:49.671 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39188.log 2024-09-18T07:18:49.671 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mon.a.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.39608.log.gz 2024-09-18T07:18:49.671 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37626.log 2024-09-18T07:18:49.671 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39188.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.39188.log.gz 2024-09-18T07:18:49.671 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39314.log 2024-09-18T07:18:49.671 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37626.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.37626.log.gz 2024-09-18T07:18:49.671 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.38870.log 2024-09-18T07:18:49.671 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39314.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.39314.log.gz 2024-09-18T07:18:49.671 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.38664.log 2024-09-18T07:18:49.676 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.libcephsqlite.38870.log: 90.4% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.38870.log.gz 2024-09-18T07:18:49.676 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-18T07:18:49.676 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.libcephsqlite.38664.log: 88.0% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.38664.log.gz 2024-09-18T07:18:49.676 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37063.log 2024-09-18T07:18:49.676 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.2.log: 95.5% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.38788.log.gz 2024-09-18T07:18:49.679 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39104.log 2024-09-18T07:18:49.679 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37063.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.37063.log.gz 2024-09-18T07:18:49.681 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31142.log 2024-09-18T07:18:49.681 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31051.log 2024-09-18T07:18:49.681 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-18T07:18:49.681 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.31142.log: /var/log/ceph/ceph-client.admin.31051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30971.log 2024-09-18T07:18:49.681 INFO:teuthology.orchestra.run.smithi172.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.31142.log.gz 2024-09-18T07:18:49.681 INFO:teuthology.orchestra.run.smithi172.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.31051.log.gz 2024-09-18T07:18:49.681 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30931.log 2024-09-18T07:18:49.681 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.30971.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-18T07:18:49.681 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.30931.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30971.log.gz 2024-09-18T07:18:49.682 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T07:18:49.682 INFO:teuthology.orchestra.run.smithi172.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.30931.log.gz 2024-09-18T07:18:49.682 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31182.log 2024-09-18T07:18:49.682 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31091.log 2024-09-18T07:18:49.682 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.31182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31222.log 2024-09-18T07:18:49.682 INFO:teuthology.orchestra.run.smithi172.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.31182.log.gz 2024-09-18T07:18:49.682 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.31091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31011.log 2024-09-18T07:18:49.682 INFO:teuthology.orchestra.run.smithi172.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T07:18:49.682 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.31222.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.31091.log.gz 2024-09-18T07:18:49.682 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-18T07:18:49.682 INFO:teuthology.orchestra.run.smithi172.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.31222.log.gz 2024-09-18T07:18:49.682 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-18T07:18:49.682 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.31011.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T07:18:49.683 INFO:teuthology.orchestra.run.smithi172.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.31011.log.gz 2024-09-18T07:18:49.683 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-18T07:18:49.683 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-18T07:18:49.683 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-mon.b.log: 86.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T07:18:49.683 INFO:teuthology.orchestra.run.smithi172.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-18T07:18:49.684 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39482.log 2024-09-18T07:18:49.685 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39104.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.39104.log.gz 2024-09-18T07:18:49.687 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31729.log 2024-09-18T07:18:49.687 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39482.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.39482.log.gz 2024-09-18T07:18:49.687 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-18T07:18:49.688 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.31729.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.31729.log.gz 2024-09-18T07:18:49.688 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39230.log 2024-09-18T07:18:49.700 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T07:18:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39230.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.39230.log.gz 2024-09-18T07:18:49.707 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37743.log 2024-09-18T07:18:49.720 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39356.log 2024-09-18T07:18:49.720 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37743.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.37743.log.gz 2024-09-18T07:18:49.721 INFO:teuthology.orchestra.run.smithi070.stderr: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T07:18:49.738 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36635.log 2024-09-18T07:18:49.738 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39356.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.39356.log.gz 2024-09-18T07:18:49.751 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.38719.log 2024-09-18T07:18:49.751 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.36635.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36635.log.gz 2024-09-18T07:18:49.753 INFO:teuthology.orchestra.run.smithi070.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-18T07:18:49.753 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36809.log 2024-09-18T07:18:49.763 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.libcephsqlite.38719.log: 94.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37407.log 2024-09-18T07:18:49.764 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.36809.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36809.log.gz 2024-09-18T07:18:49.764 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.libcephsqlite.38719.log.gz 2024-09-18T07:18:49.766 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.38830.log 2024-09-18T07:18:49.766 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37407.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.37407.log.gz 2024-09-18T07:18:49.766 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37985.log 2024-09-18T07:18:49.774 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.libcephsqlite.38830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36691.log 2024-09-18T07:18:49.775 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37985.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.37985.log.gz 2024-09-18T07:18:49.787 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37742.log 2024-09-18T07:18:49.787 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.36691.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36691.log.gz 2024-09-18T07:18:49.800 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.38739.log 2024-09-18T07:18:49.800 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37742.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.37742.log.gz 2024-09-18T07:18:49.809 INFO:teuthology.orchestra.run.smithi172.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-18T07:18:49.812 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37747.log 2024-09-18T07:18:49.812 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.libcephsqlite.38739.log: 87.8% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.38739.log.gz 2024-09-18T07:18:49.813 INFO:teuthology.orchestra.run.smithi070.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-18T07:18:49.813 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38027.log 2024-09-18T07:18:49.815 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37747.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.37747.log.gz 2024-09-18T07:18:49.815 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37072.log 2024-09-18T07:18:49.816 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38027.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.38027.log.gz 2024-09-18T07:18:49.816 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37410.log 2024-09-18T07:18:49.819 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37072.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.37072.log.gz 2024-09-18T07:18:49.819 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39818.log 2024-09-18T07:18:49.819 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37410.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.37410.log.gz 2024-09-18T07:18:49.819 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37411.log 2024-09-18T07:18:49.819 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39818.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.39818.log.gz 2024-09-18T07:18:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37070.log 2024-09-18T07:18:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37411.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.37411.log.gz 2024-09-18T07:18:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37744.log 2024-09-18T07:18:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37070.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.37070.log.gz 2024-09-18T07:18:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-18T07:18:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37744.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.37744.log.gz 2024-09-18T07:18:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.38807.log 2024-09-18T07:18:49.836 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37064.log 2024-09-18T07:18:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.libcephsqlite.38807.log: 87.9% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.38807.log.gz 2024-09-18T07:18:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.38643.log 2024-09-18T07:18:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37064.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.37064.log.gz 2024-09-18T07:18:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.38600.log 2024-09-18T07:18:49.864 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.libcephsqlite.38643.log: 87.8% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.38643.log.gz 2024-09-18T07:18:49.865 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.libcephsqlite.38600.log: 87.6% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.38600.log.gz 2024-09-18T07:18:49.895 INFO:teuthology.orchestra.run.smithi070.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.38762.log.gz 2024-09-18T07:18:49.943 INFO:teuthology.orchestra.run.smithi070.stderr: 89.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-18T07:18:50.196 INFO:teuthology.orchestra.run.smithi070.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.38830.log.gz 2024-09-18T07:18:51.246 INFO:teuthology.orchestra.run.smithi070.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.38150.log.gz 2024-09-18T07:18:52.208 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-18T07:18:52.209 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-18T07:18:52.209 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-18T07:18:52.209 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-18T07:18:52.209 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-18T07:18:52.209 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-18T07:18:52.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-18T07:18:52.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-18T07:18:52.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-09-18T07:18:52.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-18T07:18:52.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-18T07:18:54.279 INFO:teuthology.orchestra.run.smithi070.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-18T07:18:55.000 INFO:teuthology.orchestra.run.smithi172.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-18T07:19:01.876 INFO:teuthology.orchestra.run.smithi070.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-18T07:19:04.165 INFO:teuthology.orchestra.run.smithi172.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-18T07:19:05.643 INFO:teuthology.orchestra.run.smithi172.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-18T07:19:06.536 INFO:teuthology.orchestra.run.smithi070.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-18T07:19:06.915 INFO:teuthology.orchestra.run.smithi172.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-18T07:19:06.917 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-09-18T07:19:06.917 INFO:teuthology.orchestra.run.smithi172.stderr:real 0m17.255s 2024-09-18T07:19:06.918 INFO:teuthology.orchestra.run.smithi172.stderr:user 0m51.427s 2024-09-18T07:19:06.918 INFO:teuthology.orchestra.run.smithi172.stderr:sys 0m1.667s 2024-09-18T07:19:09.981 INFO:teuthology.orchestra.run.smithi070.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-18T07:19:09.983 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-09-18T07:19:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:real 0m20.358s 2024-09-18T07:19:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:user 0m54.930s 2024-09-18T07:19:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:sys 0m1.796s 2024-09-18T07:19:09.983 INFO:tasks.ceph:Archiving logs... 2024-09-18T07:19:09.984 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi/7910278/remote/smithi070/log 2024-09-18T07:19:09.985 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T07:19:18.945 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/log/ceph to /home/teuthworker/archive/nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi/7910278/remote/smithi172/log 2024-09-18T07:19:18.946 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T07:19:27.181 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-18T07:19:27.190 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-18T07:19:27.190 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T07:19:27.193 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T07:19:27.284 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-18T07:19:27.292 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-18T07:19:27.292 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T07:19:27.295 DEBUG:teuthology.orchestra.run.smithi172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T07:19:27.310 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-18T07:19:27.312 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-09-18T07:19:27.315 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T07:19:27.315 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-18T07:19:27.315 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 -1593us[-1599us] +/- 48ms 2024-09-18T07:19:27.315 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 +772us[ +765us] +/- 60ms 2024-09-18T07:19:27.315 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +1910us[+1910us] +/- 86ms 2024-09-18T07:19:27.315 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T07:19:27.317 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T07:19:27.317 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-09-18T07:19:27.317 INFO:teuthology.orchestra.run.smithi172.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 -1545us[-1548us] +/- 48ms 2024-09-18T07:19:27.317 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 +787us[ +787us] +/- 60ms 2024-09-18T07:19:27.317 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +1931us[+1931us] +/- 86ms 2024-09-18T07:19:27.317 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T07:19:27.318 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-18T07:19:27.326 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-18T07:19:27.327 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-18T07:19:27.335 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-18T07:19:27.424 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-18T07:19:27.490 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T07:19:27.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T07:19:27.520 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T07:19:27.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T07:19:27.549 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-18T07:19:27.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-18T07:19:27.567 INFO:teuthology.task.internal:Duration was 700.270049 seconds 2024-09-18T07:19:27.567 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-18T07:19:27.576 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-18T07:19:27.576 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T07:19:27.579 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T07:19:27.619 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T07:19:27.629 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T07:19:28.035 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-18T07:19:28.035 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2024-09-18T07:19:28.035 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T07:19:28.098 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi172.front.sepia.ceph.com 2024-09-18T07:19:28.098 DEBUG:teuthology.orchestra.run.smithi172:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T07:19:28.127 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-18T07:19:28.128 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T07:19:28.141 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T07:19:28.196 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-18T07:19:28.196 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T07:19:28.210 DEBUG:teuthology.orchestra.run.smithi172:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T07:19:28.335 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-18T07:19:28.346 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-18T07:19:28.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T07:19:28.372 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T07:19:28.398 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-18T07:19:28.408 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T07:19:28.415 DEBUG:teuthology.orchestra.run.smithi172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T07:19:28.435 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2024-09-18T07:19:28.461 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = core 2024-09-18T07:19:28.485 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T07:19:28.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T07:19:28.503 DEBUG:teuthology.orchestra.run.smithi172:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T07:19:28.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T07:19:28.541 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-18T07:19:28.551 INFO:teuthology.task.internal:Transferring archived files... 2024-09-18T07:19:28.552 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi/7910278/remote/smithi070 2024-09-18T07:19:28.552 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T07:19:28.586 DEBUG:teuthology.misc:Transferring archived files from smithi172:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/nmordech-2024-09-18_06:58:32-rados:basic-main-distro-default-smithi/7910278/remote/smithi172 2024-09-18T07:19:28.587 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T07:19:28.620 INFO:teuthology.task.internal:Removing archive directory... 2024-09-18T07:19:28.620 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T07:19:28.624 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T07:19:28.670 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-18T07:19:28.682 INFO:teuthology.task.internal:Not uploading archives. 2024-09-18T07:19:28.682 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-18T07:19:28.698 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-18T07:19:28.698 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T07:19:28.701 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T07:19:28.713 INFO:teuthology.orchestra.run.smithi070.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 07:19 /home/ubuntu/cephtest 2024-09-18T07:19:28.725 INFO:teuthology.orchestra.run.smithi172.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 07:19 /home/ubuntu/cephtest 2024-09-18T07:19:28.727 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-18T07:19:28.736 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-18T07:19:28.759 INFO:teuthology.run:Summary data: description: rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} tasks/libcephsqlite} duration: 700.2700493335724 flavor: default owner: scheduled_nmordech@teuthology success: true 2024-09-18T07:19:28.759 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T07:19:28.850 INFO:teuthology.run:pass